-- Logs begin at Wed 2021-11-10 03:11:39 UTC, end at Wed 2021-11-10 05:24:27 UTC. -- Nov 10 03:48:14.978769 ncdv-817176-1 systemd[1]: Started Devstack devstack@c-bak.service. Nov 10 03:48:17.996727 ncdv-817176-1 cinder-backup[127309]: INFO cinder.cmd.backup [-] Backup running with 1 processes. Nov 10 03:48:17.997710 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=127309) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Nov 10 03:48:17.997961 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=127309) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Nov 10 03:48:18.194087 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_db.sqlalchemy.engines [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127309) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 03:48:18.472976 ncdv-817176-1 cinder-backup[127309]: INFO cinder.rpc [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Nov 10 03:48:18.478875 ncdv-817176-1 cinder-backup[127309]: INFO cinder.rpc [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version. Nov 10 03:48:18.483042 ncdv-817176-1 cinder-backup[127309]: INFO cinder.rpc [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Nov 10 03:48:18.487018 ncdv-817176-1 cinder-backup[127309]: INFO cinder.rpc [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version. Nov 10 03:48:18.492024 ncdv-817176-1 cinder-backup[127309]: INFO cinder.rpc [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Nov 10 03:48:18.496277 ncdv-817176-1 cinder-backup[127309]: INFO cinder.rpc [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Nov 10 03:48:18.548718 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_db.sqlalchemy.engines [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127309) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 03:48:18.568750 ncdv-817176-1 cinder-backup[127309]: INFO oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Starting 1 workers Nov 10 03:48:18.575161 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Started child 127755 {{(pid=127309) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Nov 10 03:48:18.576622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Full set of CONF: {{(pid=127309) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Nov 10 03:48:18.577155 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] ******************************************************************************** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 10 03:48:18.577626 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] Configuration options gathered from: {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 10 03:48:18.578227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 10 03:48:18.579126 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 10 03:48:18.579741 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] ================================================================================ {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 10 03:48:18.580761 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] allow_availability_zone_fallback = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.581322 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] allow_compression_on_image_upload = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.582471 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] allowed_direct_url_schemes = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.583342 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] api_rate_limit = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.584355 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] auth_strategy = keystone {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.584937 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_availability_zone = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.585503 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_api_class = cinder.backup.api.API {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.586147 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_chunk_size = 134217728 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.586794 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_conf = /etc/ceph/ceph.conf {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.587322 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_image_journals = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.587868 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_pool = backups {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.588494 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_stripe_count = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.589040 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_stripe_unit = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.589747 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_ceph_user = cinder-bak {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.590356 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_driver = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.591059 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_driver_init_check_interval = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.591626 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_driver_stats_polling_interval = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.592482 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.593441 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_max_operations = 15 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.594520 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_metadata_version = 2 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.595075 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_name_template = backup-%s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.595644 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_native_threads_pool_size = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.596220 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_object_number_per_notification = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.596848 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_service_inithost_offload = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.597358 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_timer_interval = 120 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.597954 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_use_same_host = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.599098 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_use_temp_snapshot = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.599821 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backup_workers = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.600088 ncdv-817176-1 cinder-backup[127755]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0) Nov 10 03:48:18.601067 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] chap_password = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.601588 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] chap_username = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.602157 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.602951 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_internal_tenant_project_id = 965b7da990b947c19ba74a09cc22832f {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.603749 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_internal_tenant_user_id = 7e779dee8d114aa890e1725ef72b0ac7 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.604443 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] compression_format = gzip {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.605169 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] config_dir = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.605643 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.606243 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] config_source = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.606786 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.607436 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] control_exchange = openstack {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.608011 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] debug = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.608556 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] default_availability_zone = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.609123 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] default_group_type = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.610144 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.610730 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] default_volume_type = ceph {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.611281 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] driver_client_cert = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.611766 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] driver_client_cert_key = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.612298 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] driver_data_namespace = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.612997 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] driver_ssl_cert_path = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.613739 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] driver_ssl_cert_verify = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.615181 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] driver_use_ssl = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.615681 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] enable_new_services = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.616970 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] enable_unsupported_driver = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.617549 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] enabled_backends = ['ceph'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.618053 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] enforce_multipath_for_image_xfer = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.618547 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] filter_function = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.619088 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_api_insecure = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.619687 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_api_servers = ['http://10.222.0.13/image'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.620343 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_api_ssl_compression = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.620882 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_ca_certificates_file = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.621430 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_catalog_info = image:glance:publicURL {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.621914 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.622561 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.623265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.623979 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_num_retries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.624494 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] glance_request_timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.625153 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] goodness_function = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.625831 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] graceful_shutdown_timeout = 5 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.626360 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] group_api_class = cinder.group.api.API {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.626922 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] host = ncdv-817176-1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.627528 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iet_conf = /etc/iet/ietd.conf {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.628074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_compress_on_upload = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.628578 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_conversion_address_space_limit = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.629104 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_conversion_cpu_limit = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.629837 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.630442 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_upload_use_cinder_backend = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.631012 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_upload_use_internal_tenant = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.631517 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_volume_cache_enabled = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.632024 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_volume_cache_max_count = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.632530 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] image_volume_cache_max_size_gb = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.633214 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] instance_format = [instance: %(uuid)s] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.633924 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.634449 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iscsi_iotype = fileio {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.635088 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iscsi_secondary_ip_addresses = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.635729 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iscsi_target_flags = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.636352 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iscsi_write_cache = on {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.637189 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iser_helper = tgtadm {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.637971 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iser_ip_address = 10.222.0.13 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.638538 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iser_port = 3260 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.639110 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.639651 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_config_append = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.640171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.640831 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_dir = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.641341 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_file = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.641774 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_options = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.642350 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_rotate_interval = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.642897 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_rotate_interval_type = days {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.643739 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] log_rotation_type = none {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.644276 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.644784 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.645452 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.645985 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.646541 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.647174 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] max_age = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] max_logfile_count = 30 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] max_logfile_size_mb = 200 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] max_over_subscription_ratio = 20.0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] message_reap_interval = 86400 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] message_ttl = 2592000 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] monkey_patch = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.938987 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] monkey_patch_modules = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] my_ip = 10.222.0.13 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] no_snapshot_gb_quota = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] num_iser_scan_tries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] num_shell_tries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] num_volume_device_scan_tries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] nvmet_ns_id = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941184 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] nvmet_port_id = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941946 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_max_limit = 1000 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941946 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_volume_ext_list = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.941946 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942498 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.lockutils [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] Lock "cleanup_incomplete_backups_127309" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.001s {{(pid=127755) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Nov 10 03:48:18.942578 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_volume_listen = 0.0.0.0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942662 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] Cleaning up incomplete backup operations. Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_volume_listen_port = 8776 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_volume_use_ssl = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] osapi_volume_workers = 2 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] per_volume_size_limit = -1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] periodic_fuzzy_delay = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] periodic_interval = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.942792 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] public_endpoint = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] publish_errors = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_backup_gigabytes = 1000 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_backups = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_consistencygroups = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_gigabytes = 1000 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.943227 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_groups = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_snapshots = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] quota_volumes = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] rate_limit_burst = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] rate_limit_except_level = CRITICAL {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] rate_limit_interval = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] replication_device = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944079 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] report_discard_supported = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944587 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] report_interval = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944587 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] reservation_clean_interval = 86400 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944587 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] reservation_expire = 86400 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944587 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] reserved_percentage = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944587 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.944587 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] restore_discard_excess_bytes = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] rpc_response_timeout = 120 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] run_external_periodic_tasks = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] scst_target_driver = iscsi {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] scst_target_iqn_name = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_down_time = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] snapshot_name_template = snapshot-%s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] split_loggers = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] state_path = /opt/stack/data/cinder {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] storage_availability_zone = nova {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] storage_protocol = iscsi {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.945818 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] syslog_log_facility = LOG_USER {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] target_helper = lioadm {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] target_ip_address = 10.222.0.13 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] target_port = 3260 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] target_protocol = iscsi {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946265 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] trace_flags = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] transfer_api_class = cinder.transfer.api.API {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] transport_url = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] until_refresh = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_chap_auth = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_default_quota_class = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_eventlog = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.946622 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_journal = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_json = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_multipath_for_image_xfer = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_stderr = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] use_syslog = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] verify_glance_signatures = enabled {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_api_class = cinder.volume.api.API {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_backend_name = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947973 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_clear = zero {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947973 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_clear_ionice = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947973 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_clear_size = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947973 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947973 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_copy_bps_limit = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.947973 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_dd_blocksize = 1M {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.948346 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.948346 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_name_template = volume-%s {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.948346 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volume_usage_audit_period = month {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.948346 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.948346 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] watch_log_file = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 10 03:48:18.948346 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949531 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.enforce_new_defaults = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949531 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.enforce_scope = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949531 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.policy_default_rule = default {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949531 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949531 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.policy_file = policy.yaml {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949531 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949892 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949892 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949892 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949892 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949892 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_concurrency.disable_process_locking = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.949892 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.admin_password = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.admin_token = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.admin_user = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_admin_prefix = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950257 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_port = 35357 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950694 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_protocol = https {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950694 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_section = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950694 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_type = password {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950694 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_uri = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950694 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.auth_version = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.950694 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.cache = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.delay_auth_decision = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.http_connect_timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951171 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951624 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.identity_uri = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951624 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.include_service_catalog = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951624 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.insecure = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951624 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.interface = public {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951624 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951624 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951988 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951988 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951988 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951988 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951988 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.951988 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.952393 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.952393 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.952393 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.region_name = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.952393 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.952393 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.service_token_roles_required = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.952393 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.service_type = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953101 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.token_cache_time = 300 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953101 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953101 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.backend = dogpile.cache.null {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953101 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.backend_argument = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953101 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.config_prefix = cache.oslo {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953101 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.debug_cache_backend = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.enabled = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.expiration_time = 600 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_dead_retry = 300 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_pool_maxsize = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953489 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.memcache_socket_timeout = 1.0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.proxies = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.tls_allowed_ciphers = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.tls_cafile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.tls_certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.953936 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.tls_enabled = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cache.tls_keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] coordination.backend_url = etcd3+http://10.222.0.13:2379 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.backend = sqlalchemy {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.connection = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.connection_debug = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.connection_parameters = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954336 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.connection_recycle_time = 3600 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.connection_trace = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.db_inc_retry_interval = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.db_max_retries = 20 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.db_max_retry_interval = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.db_retry_interval = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.max_overflow = 50 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.954764 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.max_pool_size = 5 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.max_retries = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.mysql_enable_ndb = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.pool_timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.retry_interval = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.slave_connection = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955446 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.sqlite_synchronous = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955864 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] database.use_db_reconnect = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955864 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955864 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955864 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955864 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.955864 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.connection_string = messaging:// {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.enabled = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.es_doc_type = notification {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.es_scroll_size = 10000 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.es_scroll_time = 2m {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.filter_error_trace = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956307 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.hmac_keys = SECRET_KEY {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.sentinel_service_name = mymaster {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.socket_timeout = 0.1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] profiler.trace_sqlalchemy = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] privsep_osbrick.capabilities = [21] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] privsep_osbrick.group = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] privsep_osbrick.helper_command = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.956799 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] privsep_osbrick.user = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_sys_admin.group = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_sys_admin.helper_command = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957285 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] cinder_sys_admin.user = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.auth_section = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.auth_type = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.cafile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.collect_timing = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.957688 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.insecure = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.send_service_user_token = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.split_loggers = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] service_user.timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.backend_availability_zone = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.chap_password = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.958640 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.chap_username = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.959813 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.959813 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.959813 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.959813 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.driver_client_cert = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.959813 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.driver_client_cert_key = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.959813 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.driver_data_namespace = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960246 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960246 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960246 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.driver_use_ssl = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960246 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.enable_deferred_deletion = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960246 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.enable_unsupported_driver = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960246 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960654 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.filter_function = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960654 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.goodness_function = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960654 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960654 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960654 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.960654 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iscsi_iotype = fileio {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iscsi_target_flags = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961074 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iscsi_write_cache = on {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iser_helper = tgtadm {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iser_ip_address = 10.222.0.13 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iser_port = 3260 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961526 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.num_shell_tries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.nvmet_port_id = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rados_connection_interval = 5 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.961906 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rados_connection_retries = 3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.962325 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_ceph_conf = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.962325 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.962325 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.962325 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.962325 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.962325 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_pool = rbd {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966134 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_secret_uuid = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966134 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966134 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.rbd_user = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966134 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966134 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.replication_device = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966134 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.report_discard_supported = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966510 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966510 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.reserved_percentage = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966510 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.scst_target_driver = iscsi {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966510 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.scst_target_iqn_name = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966510 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.storage_protocol = iscsi {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966510 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.target_helper = tgtadm {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966889 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.target_ip_address = 10.222.0.13 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966889 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.target_port = 3260 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966889 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966889 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.target_protocol = iscsi {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966889 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.trace_flags = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.966889 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.unique_fqdn_network = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967311 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.use_chap_auth = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967311 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967311 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_backend_name = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967311 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_clear = zero {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967311 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_clear_ionice = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967311 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_clear_size = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967731 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967731 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967731 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967731 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967731 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_reports.file_event_handler = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.967731 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.968148 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_reports.log_dir = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.968148 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.968148 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.968148 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.968148 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.968148 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969539 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969539 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969539 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969539 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969539 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969539 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969903 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969903 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969903 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969903 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969903 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.969903 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970261 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970261 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970261 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970261 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970261 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.ssl = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970261 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970616 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970616 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970616 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_rabbit.ssl_version = {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970616 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_notifications.driver = [] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970616 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_notifications.retry = -1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.970616 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971001 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] oslo_messaging_notifications.transport_url = **** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971001 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971001 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971001 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.barbican_api_version = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971001 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.barbican_endpoint = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971001 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.barbican_endpoint_type = public {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971359 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.barbican_region_name = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971359 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.cafile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971359 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971359 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.collect_timing = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.971359 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.insecure = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.984976 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.number_of_retries = 60 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.retry_delay = 1 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.send_service_user_token = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.split_loggers = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985070 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.verify_ssl = True {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican.verify_ssl_path = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.auth_section = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.auth_type = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.cafile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.collect_timing = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985469 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.insecure = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.split_loggers = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] barbican_service_user.timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.approle_role_id = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.approle_secret_id = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.cafile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.985989 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.certfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.collect_timing = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.insecure = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.keyfile = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.kv_mountpoint = secret {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.kv_version = 2 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.namespace = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986533 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.root_token_id = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986972 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.split_loggers = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986972 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.ssl_ca_crt_file = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986972 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.timeout = None {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986972 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.use_ssl = False {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986972 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 10 03:48:18.986972 ncdv-817176-1 cinder-backup[127309]: DEBUG oslo_service.service [None req-452f41f5-6b5a-49cd-b39a-92c6a17a6ca3 None None] ******************************************************************************** {{(pid=127309) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 10 03:48:19.019695 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.lockutils [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] Lock "cleanup_incomplete_backups_127309" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.365s {{(pid=127755) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Nov 10 03:48:19.029855 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 03:48:19.037728 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:48:19.073448 ncdv-817176-1 cinder-backup[127755]: INFO cinder.keymgr.migration [None req-c18b2be7-2f47-4eee-8d22-dea7317be59e None None] Not migrating encryption keys because the ConfKeyManager is still in use. Nov 10 03:48:19.106931 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.service [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] Creating RPC server for service cinder-backup {{(pid=127755) start /opt/stack/cinder/cinder/service.py:231}} Nov 10 03:48:19.128414 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.service [None req-e6e2aa53-f762-47fb-a9f1-e8f097079461 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=127755) start /opt/stack/cinder/cinder/service.py:237}} Nov 10 03:49:26.177693 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:49:26.178393 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:50:26.172749 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:50:26.173376 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:51:26.177765 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:51:26.178407 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:52:26.183289 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:52:26.183875 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:53:26.188539 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:53:26.189148 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:54:26.194327 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:54:26.194925 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:55:26.200087 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:55:26.200673 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:56:26.205188 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:56:26.206254 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:58:26.210522 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:58:26.214415 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 03:59:26.206120 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 03:59:26.207259 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:00:26.205539 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:00:26.206136 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:01:26.210639 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:01:26.211333 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:02:26.207718 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:02:26.208312 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:03:26.213323 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:03:26.214556 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:04:26.209462 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:04:26.210194 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:05:26.209924 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:05:26.210981 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:06:26.211371 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:06:26.212027 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:07:26.210629 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:07:26.211282 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:08:26.216637 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:08:26.217191 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:09:26.221994 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:09:26.222599 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:10:26.221395 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:10:26.222850 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:11:26.223372 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:11:26.224131 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:12:14.553923 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Create backup started, backup: 070eb9cb-72ec-482f-9cc9-95cbb02dc772 volume: 10e45ab1-9a66-45b6-a400-7c909cbde2b1 snapshot: d13be209-8612-4808-95ed-428e85a1de22. Nov 10 04:12:14.565946 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:12:14.567918 ncdv-817176-1 cinder-backup[127755]: INFO oslo.privsep.daemon [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpsvf0wigi/privsep.sock'] Nov 10 04:12:14.698663 ncdv-817176-1 sudo[150457]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpsvf0wigi/privsep.sock Nov 10 04:12:14.710256 ncdv-817176-1 sudo[150457]: pam_unix(sudo:session): session opened for user root by (uid=0) Nov 10 04:12:16.866576 ncdv-817176-1 sudo[150457]: pam_unix(sudo:session): session closed for user root Nov 10 04:12:16.871524 ncdv-817176-1 cinder-backup[127755]: INFO oslo.privsep.daemon [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Spawned new privsep daemon via rootwrap Nov 10 04:12:16.872166 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Accepted privsep connection to /tmp/tmpsvf0wigi/privsep.sock {{(pid=127755) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}} Nov 10 04:12:16.873932 ncdv-817176-1 cinder-backup[127755]: INFO oslo.privsep.daemon [-] privsep daemon starting Nov 10 04:12:16.874206 ncdv-817176-1 cinder-backup[127755]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Nov 10 04:12:16.874494 ncdv-817176-1 cinder-backup[127755]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none Nov 10 04:12:16.874752 ncdv-817176-1 cinder-backup[127755]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 150467 Nov 10 04:12:16.876339 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[30a49b98-9f0c-4b27-9239-3bdf9fdf742f]: (2,) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.133504 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.154221 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.021s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:12:17.155535 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[c806b884-99a7-40b0-b168-c8cb83ea60fc]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.159202 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.167240 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[08d09185-140a-441a-bfff-2c6e21d141d2]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:12:17.192047 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:12:17.193902 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[08d09185-140a-441a-bfff-2c6e21d141d2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.198843 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.205924 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a3e56edf-4d5a-4c01-9ae3-d2b5c7e6b81d]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:12:17.206292 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:12:17.207510 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3e56edf-4d5a-4c01-9ae3-d2b5c7e6b81d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.213332 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.240776 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.030s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:12:17.241145 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[0bda58c8-1e49-4923-8b70-9295bd8b46cb]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.242585 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.311324 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.068s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:12:17.312033 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[3789b93d-9ea1-446c-afb9-2d00e15374fe]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.313602 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.336983 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.023s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:12:17.337981 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d9fcf8a-0b43-408e-8a5e-eeedca2ab4f0]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:12:17.339753 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:12:17.391446 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] CMD "nvme version" returned: 0 in 0.052s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:12:17.397311 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] <== get_connector_properties: return (2829ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:12:18.498694 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:12:18.503442 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-25d7a4ab-a882-45db-846a-1a02867fed3f', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '25d7a4ab-a882-45db-846a-1a02867fed3f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:12:18.506633 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:12:18.605792 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] <== connect_volume: return (101ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:12:18.615157 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:12:18.640515 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Starting backup of volume='10e45ab1-9a66-45b6-a400-7c909cbde2b1'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:12:18.675381 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:12:18.675785 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Creating backup base image='volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.base' for volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1. {{(pid=127755) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}} Nov 10 04:12:20.601024 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Copying data from volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1. {{(pid=127755) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}} Nov 10 04:12:20.634069 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferring data between 'volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1' and 'volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.base' {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}} Nov 10 04:12:20.634459 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] 8 chunks of 134217728 bytes to be transferred {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}} Nov 10 04:12:22.309365 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 1 of 8 (78320K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:23.640782 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 2 of 8 (98433K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:24.679698 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 3 of 8 (126230K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:25.957252 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 4 of 8 (102694K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:26.228104 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:12:26.228815 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:12:27.242729 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 5 of 8 (102009K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:28.623728 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 6 of 8 (94995K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:30.052078 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 7 of 8 (92030K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:31.738090 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Transferred chunk 8 of 8 (77668K/s) {{(pid=127755) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 10 04:12:31.787904 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:12:31.806083 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:12:31.817893 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:12:31.850489 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:12:31.851074 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:12:31.861018 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] No metadata type 'volume-glance-metadata' available {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 10 04:12:31.861580 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Backing up metadata for volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:12:31.934404 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Backup '070eb9cb-72ec-482f-9cc9-95cbb02dc772' of volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1 finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:12:31.936923 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-25d7a4ab-a882-45db-846a-1a02867fed3f', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '25d7a4ab-a882-45db-846a-1a02867fed3f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:12:31.964136 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] <== disconnect_volume: return (26ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:12:32.151882 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-d9d748c1-737f-4372-9176-79a061a96aab tempest-VolumesSnapshotTestJSON-106401922 None] Create backup finished. backup: 070eb9cb-72ec-482f-9cc9-95cbb02dc772. Nov 10 04:12:33.281486 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Delete backup started, backup: 070eb9cb-72ec-482f-9cc9-95cbb02dc772. Nov 10 04:12:33.284677 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Delete started for backup=070eb9cb-72ec-482f-9cc9-95cbb02dc772 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:12:33.285434 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Trying diff format basename='volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.070eb9cb-72ec-482f-9cc9-95cbb02dc772' for backup base image of volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:12:33.344019 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Image 'volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.070eb9cb-72ec-482f-9cc9-95cbb02dc772' not found - trying diff format name {{(pid=127755) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}} Nov 10 04:12:33.384201 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Looking for snapshot of backup base 'volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.base' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:12:33.384974 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Backup base 'volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.base' has no snapshots {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}} Nov 10 04:12:33.385499 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] No backup snapshot to delete {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Nov 10 04:12:33.401844 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Deleting backup base image='volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.base' of volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1. Nov 10 04:12:34.516235 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Base backup image='volume-10e45ab1-9a66-45b6-a400-7c909cbde2b1.backup.base' of volume 10e45ab1-9a66-45b6-a400-7c909cbde2b1 deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:12:34.588884 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Delete of backup '070eb9cb-72ec-482f-9cc9-95cbb02dc772' for volume '10e45ab1-9a66-45b6-a400-7c909cbde2b1' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:12:34.593383 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:12:34.648252 ncdv-817176-1 cinder-backup[127755]: WARNING cinder.quota [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Deprecated: Default quota for resource: backups is set by the default quota flag: quota_backups, it is now deprecated. Please use the default quota class for default quota. Nov 10 04:12:34.649098 ncdv-817176-1 cinder-backup[127755]: WARNING cinder.quota [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Deprecated: Default quota for resource: backup_gigabytes is set by the default quota flag: quota_backup_gigabytes, it is now deprecated. Please use the default quota class for default quota. Nov 10 04:12:34.691580 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Created reservations ['40d26456-8d21-48ae-a4a1-0186a248c845', 'd43bbc55-7814-4bf0-a496-9330b7e13ebf'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:12:34.748847 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-cc6f921c-bdba-4222-afa1-7d9c4fb05737 tempest-VolumesSnapshotTestJSON-106401922 None] Delete backup finished, backup 070eb9cb-72ec-482f-9cc9-95cbb02dc772 deleted. Nov 10 04:13:26.230091 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:13:26.230729 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:14:26.229374 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:14:26.230055 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:15:26.230760 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:15:26.231370 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:16:26.230999 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:16:26.231604 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:17:26.226811 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:17:26.227425 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:18:26.227821 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:18:26.228558 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:19:26.223854 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:19:26.225019 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:19:59.819004 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Create backup started, backup: fa7fe21a-9848-4933-a96b-b4e065c66873 volume: 42d63a60-be1b-40a5-b58e-65bd01edd547. Nov 10 04:19:59.845116 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:19:59.846776 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:19:59.878643 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.029s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:19:59.879634 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[328028ed-596f-4643-8af9-e04cef852ce4]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:19:59.885639 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:19:59.900740 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[fadc204f-6974-44af-948d-1163dd2f2c9a]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:19:59.901101 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:19:59.902472 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[fadc204f-6974-44af-948d-1163dd2f2c9a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:19:59.905824 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:19:59.918863 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[73f7b583-fbf0-4fa4-9e92-ec3c3f7c4e0b]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:19:59.920328 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:19:59.921651 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[73f7b583-fbf0-4fa4-9e92-ec3c3f7c4e0b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:19:59.925476 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:19:59.971046 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.047s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:19:59.971542 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[85aa8154-947d-4c1f-88e3-9e17415f06f0]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:19:59.974034 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:00.033757 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.059s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:00.034802 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5b81f5f-e4e4-4dee-858c-5a1cb878a831]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:00.038059 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:00.075200 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.037s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:00.075754 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[1967229e-7222-409a-a75e-2431ddffdfd1]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:00.076645 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:00.129182 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] CMD "nvme version" returned: 0 in 0.051s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:00.130648 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] <== get_connector_properties: return (284ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:20:01.887928 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:20:01.889517 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-42d63a60-be1b-40a5-b58e-65bd01edd547', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '42d63a60-be1b-40a5-b58e-65bd01edd547', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:20:01.891364 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:20:02.033965 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] <== connect_volume: return (143ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:20:02.045692 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:20:02.092519 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Starting backup of volume='42d63a60-be1b-40a5-b58e-65bd01edd547'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:20:02.094430 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Volume file is RBD: attempting optimized backup {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 10 04:20:02.134495 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Creating base image 'volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' {{(pid=127755) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 10 04:20:02.134994 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:20:02.207664 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Using --from-snap 'None' for incremental backup of volume 42d63a60-be1b-40a5-b58e-65bd01edd547. {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 10 04:20:02.208453 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Creating backup snapshot='backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071' {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 10 04:20:02.819432 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Performing differential transfer from 'volume-42d63a60-be1b-40a5-b58e-65bd01edd547' to 'volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:20:02.820249 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd__fxa9gdk --pool volumes volumes/volume-42d63a60-be1b-40a5-b58e-65bd01edd547@backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:20:02.821600 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:20:11.791343 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Differential backup transfer completed in 8.9704s {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 10 04:20:11.792561 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:20:11.819465 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:20:11.822002 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:20:11.847096 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:20:11.847704 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:20:11.869926 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 10 04:20:11.873988 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Backing up metadata for volume 42d63a60-be1b-40a5-b58e-65bd01edd547. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:20:11.967161 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Backup 'fa7fe21a-9848-4933-a96b-b4e065c66873' of volume 42d63a60-be1b-40a5-b58e-65bd01edd547 finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:20:11.969948 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-42d63a60-be1b-40a5-b58e-65bd01edd547', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '42d63a60-be1b-40a5-b58e-65bd01edd547', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:20:12.011819 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] <== disconnect_volume: return (38ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:20:12.116439 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-3c5e97ee-17cf-495c-a510-aacf23389205 tempest-TestVolumeBackupRestore-697980072 None] Create backup finished. backup: fa7fe21a-9848-4933-a96b-b4e065c66873. Nov 10 04:20:13.678338 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Restore backup started, backup: fa7fe21a-9848-4933-a96b-b4e065c66873 volume: b93f994a-1292-4d70-ba76-4630c86e4605. Nov 10 04:20:13.709176 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:20:13.714826 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.739701 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.027s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:13.741282 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[193e70ec-7567-4e54-923e-7572088ef7b9]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:13.743177 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.750759 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[41fcb9e1-6e98-4bbb-b2a6-cc48c646f4a5]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:20:13.751565 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:20:13.752680 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[41fcb9e1-6e98-4bbb-b2a6-cc48c646f4a5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:13.753691 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.767255 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[24ea3014-d94e-4f9a-8ae2-c35b8a22c0e6]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:20:13.769014 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:20:13.770680 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[24ea3014-d94e-4f9a-8ae2-c35b8a22c0e6]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:13.775229 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.800450 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.027s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:13.801121 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[3242740f-0336-4e7a-85f8-cf62c8374863]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:13.804842 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.834911 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.031s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:13.835116 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f1ca682-b8ad-42f6-abb9-07239ec40de3]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:13.838297 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.858070 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.021s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:13.859071 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[addc3523-b5a9-428c-8561-7870d39a8e23]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:20:13.860065 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:20:13.911804 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] CMD "nvme version" returned: 0 in 0.052s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:20:13.913062 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] <== get_connector_properties: return (202ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:20:14.935957 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:20:14.937449 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-b93f994a-1292-4d70-ba76-4630c86e4605', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b93f994a-1292-4d70-ba76-4630c86e4605', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:20:14.938495 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:20:15.023269 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] <== connect_volume: return (85ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:20:15.031518 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:20:15.067882 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Starting restore from Ceph backup=fa7fe21a-9848-4933-a96b-b4e065c66873 to volume=volume-b93f994a-1292-4d70-ba76-4630c86e4605 {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 10 04:20:15.204487 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Looking for snapshot of backup base 'volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:20:15.205044 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Found snapshot 'backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:20:15.211704 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Volume file is RBD. {{(pid=127755) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 10 04:20:15.282033 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Attempting differential restore. {{(pid=127755) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 10 04:20:15.282671 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Attempting incremental restore from base='volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' snap='backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071' {{(pid=127755) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 10 04:20:15.283387 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Performing differential transfer from 'volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' to 'volume-b93f994a-1292-4d70-ba76-4630c86e4605' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:20:15.284446 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873@backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:20:15.284950 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_6yy97rjp --pool volumes - volumes/volume-b93f994a-1292-4d70-ba76-4630c86e4605' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:20:22.059011 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Restore transfer completed in 6.7755s {{(pid=127755) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 10 04:20:22.060541 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 10 04:20:22.126605 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Restoring volume base metadata {{(pid=127755) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 10 04:20:22.148730 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] No metadata of type 'volume-metadata' to restore {{(pid=127755) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 10 04:20:22.149347 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Restoring volume glance metadata {{(pid=127755) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 10 04:20:22.368121 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Restore to volume b93f994a-1292-4d70-ba76-4630c86e4605 finished successfully. {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 10 04:20:22.370210 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-b93f994a-1292-4d70-ba76-4630c86e4605', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b93f994a-1292-4d70-ba76-4630c86e4605', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:20:22.385720 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] <== disconnect_volume: return (14ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:20:22.446047 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.manager [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Encryption key ID for volume b93f994a-1292-4d70-ba76-4630c86e4605 already matches encryption key ID in backup fa7fe21a-9848-4933-a96b-b4e065c66873. {{(pid=127755) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 10 04:20:22.490187 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-51b31998-6cdc-4beb-82d4-86fd54aa5bc1 tempest-TestVolumeBackupRestore-697980072 None] Finished restoring backup fa7fe21a-9848-4933-a96b-b4e065c66873 to volume b93f994a-1292-4d70-ba76-4630c86e4605. Nov 10 04:20:26.221173 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:20:26.222211 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:21:26.223292 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:21:26.224156 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:21:52.731074 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Delete backup started, backup: fa7fe21a-9848-4933-a96b-b4e065c66873. Nov 10 04:21:52.734176 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Delete started for backup=fa7fe21a-9848-4933-a96b-b4e065c66873 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:21:52.734866 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Trying diff format basename='volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' for backup base image of volume 42d63a60-be1b-40a5-b58e-65bd01edd547. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:21:52.892377 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Looking for snapshot of backup base 'volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:21:52.896938 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Found snapshot 'backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:21:52.898692 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Deleting backup snapshot='backup.fa7fe21a-9848-4933-a96b-b4e065c66873.snap.1636518002.208071' {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 10 04:21:53.939128 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Deleting backup base image='volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' of volume 42d63a60-be1b-40a5-b58e-65bd01edd547. Nov 10 04:21:55.118873 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Base backup image='volume-42d63a60-be1b-40a5-b58e-65bd01edd547.backup.fa7fe21a-9848-4933-a96b-b4e065c66873' of volume 42d63a60-be1b-40a5-b58e-65bd01edd547 deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:21:55.200094 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Delete of backup 'fa7fe21a-9848-4933-a96b-b4e065c66873' for volume '42d63a60-be1b-40a5-b58e-65bd01edd547' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:21:55.210277 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:21:55.345426 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Created reservations ['06539538-b6ba-49b0-b75c-024de0613c28', '22d5e486-97c6-4026-81bf-4cf086ae60a7'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:21:55.400250 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-2742ea36-6e97-4b1d-8036-84c7a3d5f347 tempest-TestVolumeBackupRestore-697980072 None] Delete backup finished, backup fa7fe21a-9848-4933-a96b-b4e065c66873 deleted. Nov 10 04:22:26.228313 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:22:26.230093 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:23:21.552645 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Create backup started, backup: 0e46dec9-3678-4e16-b049-e7059603d5f9 volume: e2f104b1-3179-4960-b1ac-10e3767ce5ad. Nov 10 04:23:21.562561 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:21.564062 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.575068 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.010s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:21.575420 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5f85f8d-8fa6-4664-a5ac-058cb224efc8]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:21.578156 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.591630 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[38d50eb5-eb31-48ec-bd80-7191c516ac94]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:23:21.592123 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:23:21.593425 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[38d50eb5-eb31-48ec-bd80-7191c516ac94]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:21.593704 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.599700 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a6917219-7c84-45fb-9904-e9b077ad10cb]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:23:21.600815 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:23:21.602923 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6917219-7c84-45fb-9904-e9b077ad10cb]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:21.605276 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.616507 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.011s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:21.616914 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4d81a95-4604-40ba-b6c0-1f4aa64cb083]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:21.618351 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.630756 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.012s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:21.631868 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[150bdd94-b0ed-4048-a766-0ea0db996449]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:21.633226 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.642059 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:21.642431 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[42b6d28f-e014-4926-8d25-21e8da6d6252]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:21.643670 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:21.666427 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] CMD "nvme version" returned: 0 in 0.022s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:21.668190 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] <== get_connector_properties: return (104ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:22.607315 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:23:22.610900 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e2f104b1-3179-4960-b1ac-10e3767ce5ad', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:22.613394 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:23:22.704030 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] <== connect_volume: return (92ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:22.711578 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:23:22.741561 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Starting backup of volume='e2f104b1-3179-4960-b1ac-10e3767ce5ad'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:23:22.743225 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Volume file is RBD: attempting optimized backup {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 10 04:23:22.787165 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Creating base image 'volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' {{(pid=127755) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 10 04:23:22.787945 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:23:22.843236 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Using --from-snap 'None' for incremental backup of volume e2f104b1-3179-4960-b1ac-10e3767ce5ad. {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 10 04:23:22.843640 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Creating backup snapshot='backup.0e46dec9-3678-4e16-b049-e7059603d5f9.snap.1636518202.8432622' {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 10 04:23:23.788152 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Performing differential transfer from 'volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad' to 'volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:23:23.790606 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_j630qp61 --pool volumes volumes/volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad@backup.0e46dec9-3678-4e16-b049-e7059603d5f9.snap.1636518202.8432622 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:23:23.792015 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:23:24.864565 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Differential backup transfer completed in 1.0746s {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 10 04:23:24.866184 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:23:24.891554 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:23:24.891978 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:23:24.898805 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:23:24.899191 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:23:24.906816 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] No metadata type 'volume-glance-metadata' available {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 10 04:23:24.907609 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Backing up metadata for volume e2f104b1-3179-4960-b1ac-10e3767ce5ad. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:23:24.970122 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Backup '0e46dec9-3678-4e16-b049-e7059603d5f9' of volume e2f104b1-3179-4960-b1ac-10e3767ce5ad finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:23:24.972011 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e2f104b1-3179-4960-b1ac-10e3767ce5ad', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:24.991431 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] <== disconnect_volume: return (17ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:25.088082 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-fc5ef97d-e65c-4d1a-8420-01cd896044ee tempest-VolumesBackupsTest-1397900487 None] Create backup finished. backup: 0e46dec9-3678-4e16-b049-e7059603d5f9. Nov 10 04:23:25.816239 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Delete backup started, backup: 0e46dec9-3678-4e16-b049-e7059603d5f9. Nov 10 04:23:25.818139 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Delete started for backup=0e46dec9-3678-4e16-b049-e7059603d5f9 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:23:25.818557 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Trying diff format basename='volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' for backup base image of volume e2f104b1-3179-4960-b1ac-10e3767ce5ad. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:23:25.892549 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Looking for snapshot of backup base 'volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:23:25.893189 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Found snapshot 'backup.0e46dec9-3678-4e16-b049-e7059603d5f9.snap.1636518202.8432622' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:23:25.893608 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Deleting backup snapshot='backup.0e46dec9-3678-4e16-b049-e7059603d5f9.snap.1636518202.8432622' {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 10 04:23:26.230748 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:23:26.231219 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:23:26.901504 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Deleting backup base image='volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' of volume e2f104b1-3179-4960-b1ac-10e3767ce5ad. Nov 10 04:23:27.240810 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Base backup image='volume-e2f104b1-3179-4960-b1ac-10e3767ce5ad.backup.0e46dec9-3678-4e16-b049-e7059603d5f9' of volume e2f104b1-3179-4960-b1ac-10e3767ce5ad deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:23:27.304908 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Delete of backup '0e46dec9-3678-4e16-b049-e7059603d5f9' for volume 'e2f104b1-3179-4960-b1ac-10e3767ce5ad' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:23:27.310828 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:23:27.345698 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Created reservations ['e3455902-4f9e-408f-a344-6cddccf56726', '0f7ea1e2-1696-4260-9117-cdf511fd5a35'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:23:27.387227 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-4f430ee7-9ee7-4328-972b-81933cdc8840 tempest-VolumesBackupsTest-1397900487 None] Delete backup finished, backup 0e46dec9-3678-4e16-b049-e7059603d5f9 deleted. Nov 10 04:23:38.684726 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Create backup started, backup: 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7 volume: 6c30d2ca-7042-44c8-b348-092a2c517edb. Nov 10 04:23:38.701286 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:38.705697 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.715436 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:38.715826 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5879a96-73bf-439c-ba19-f135130a4b64]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:38.717135 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.725413 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b67a1b6c-9c0f-4b51-8983-a1d4f2830acf]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:23:38.726293 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:23:38.727619 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[b67a1b6c-9c0f-4b51-8983-a1d4f2830acf]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:38.728480 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.734315 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e36473fe-d5d4-4557-9a23-d8907e1f6628]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:23:38.734863 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:23:38.736245 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[e36473fe-d5d4-4557-9a23-d8907e1f6628]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:38.736980 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.746391 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:38.746868 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[08f1003f-5302-45a5-b66a-6bdeeebbaa46]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:38.749774 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.756217 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:38.756813 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[75f1c045-eb05-4d62-945f-ee1de70ef104]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:38.758350 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.765801 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:38.766145 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6fcc8a6-d2fa-4a88-a363-20710328189b]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:38.766951 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:38.785360 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:38.786449 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] <== get_connector_properties: return (84ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:39.419530 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-2d587237-930c-422a-8d75-89b95d593445 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:23:40.177469 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:23:40.178861 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-6c30d2ca-7042-44c8-b348-092a2c517edb', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6c30d2ca-7042-44c8-b348-092a2c517edb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:40.179894 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:23:40.296133 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] <== connect_volume: return (116ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:40.334708 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Starting backup of volume='6c30d2ca-7042-44c8-b348-092a2c517edb'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:23:40.336306 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Volume file is RBD: attempting optimized backup {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 10 04:23:40.396079 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Creating base image 'volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' {{(pid=127755) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 10 04:23:40.397242 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:23:40.456163 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Using --from-snap 'None' for incremental backup of volume 6c30d2ca-7042-44c8-b348-092a2c517edb. {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 10 04:23:40.456696 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Creating backup snapshot='backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949' {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 10 04:23:41.082999 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Performing differential transfer from 'volume-6c30d2ca-7042-44c8-b348-092a2c517edb' to 'volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:23:41.084224 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_cu9xr0iy --pool volumes volumes/volume-6c30d2ca-7042-44c8-b348-092a2c517edb@backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:23:41.084828 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:23:46.846386 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Differential backup transfer completed in 5.7629s {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 10 04:23:46.847840 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:23:46.877729 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:23:46.880058 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:23:46.908010 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:23:46.908396 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:23:46.928478 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 10 04:23:46.931316 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Backing up metadata for volume 6c30d2ca-7042-44c8-b348-092a2c517edb. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:23:47.000445 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Backup '2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' of volume 6c30d2ca-7042-44c8-b348-092a2c517edb finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:23:47.002214 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-6c30d2ca-7042-44c8-b348-092a2c517edb', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6c30d2ca-7042-44c8-b348-092a2c517edb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:47.036986 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] <== disconnect_volume: return (33ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:47.123544 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-98be2b57-32e9-453b-82ea-f3e1d1098768 tempest-VolumesBackupsTest-1397900487 None] Create backup finished. backup: 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7. Nov 10 04:23:49.782267 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Restore backup started, backup: 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7 volume: 4515a9a8-0c73-4773-a967-98fdff017ce5. Nov 10 04:23:49.817564 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:49.825013 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:49.854009 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.032s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:49.854975 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd5c34c5-2993-4d65-9458-25e25e0f67b3]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:49.858238 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:49.865760 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[25922217-342a-4520-9d40-db4f4248d647]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:23:49.866662 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:23:49.867948 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[25922217-342a-4520-9d40-db4f4248d647]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:49.870675 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:49.886521 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8124689b-e81d-4f54-bc1a-0f6b77141dea]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:23:49.887107 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:23:49.888535 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[8124689b-e81d-4f54-bc1a-0f6b77141dea]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:49.889393 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:49.913519 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.024s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:49.913831 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0c2a302-b3dd-4742-abfb-ed3b8ba67323]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:49.915101 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:49.946454 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.030s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:49.946958 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[a96103b4-fda5-4b6a-b62b-4d2f954a4e3e]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:49.950669 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:49.973289 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.023s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:49.973951 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fe707f0-12bd-4451-a073-84c320b9fc34]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:23:49.975018 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:23:50.027359 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] CMD "nvme version" returned: 0 in 0.051s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:23:50.029266 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] <== get_connector_properties: return (207ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:51.111918 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:23:51.116875 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-4515a9a8-0c73-4773-a967-98fdff017ce5', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '4515a9a8-0c73-4773-a967-98fdff017ce5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:51.118811 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:23:51.228133 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] <== connect_volume: return (108ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:51.237310 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:23:51.259577 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Starting restore from Ceph backup=2dfae63e-fb1b-4968-9870-e7af4b7fa2e7 to volume=volume-4515a9a8-0c73-4773-a967-98fdff017ce5 {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 10 04:23:51.417921 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Looking for snapshot of backup base 'volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:23:51.418307 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Found snapshot 'backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:23:51.427066 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Volume file is RBD. {{(pid=127755) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 10 04:23:51.479738 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Attempting differential restore. {{(pid=127755) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 10 04:23:51.480796 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Attempting incremental restore from base='volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' snap='backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949' {{(pid=127755) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 10 04:23:51.481302 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Performing differential transfer from 'volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' to 'volume-4515a9a8-0c73-4773-a967-98fdff017ce5' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:23:51.481793 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7@backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:23:51.482415 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_9jd5y8uo --pool volumes - volumes/volume-4515a9a8-0c73-4773-a967-98fdff017ce5' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:23:57.097232 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Restore transfer completed in 5.6151s {{(pid=127755) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 10 04:23:57.099276 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 10 04:23:57.157979 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Restoring volume base metadata {{(pid=127755) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 10 04:23:57.202035 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] No metadata of type 'volume-metadata' to restore {{(pid=127755) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 10 04:23:57.202476 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Restoring volume glance metadata {{(pid=127755) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 10 04:23:57.422456 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Restore to volume 4515a9a8-0c73-4773-a967-98fdff017ce5 finished successfully. {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 10 04:23:57.424877 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-4515a9a8-0c73-4773-a967-98fdff017ce5', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '4515a9a8-0c73-4773-a967-98fdff017ce5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:23:57.440267 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] <== disconnect_volume: return (14ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:23:57.508469 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.manager [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Encryption key ID for volume 4515a9a8-0c73-4773-a967-98fdff017ce5 already matches encryption key ID in backup 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7. {{(pid=127755) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 10 04:23:57.552051 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-d77707d8-9a6d-4165-84c0-56c0861ad452 tempest-VolumesBackupsTest-1397900487 None] Finished restoring backup 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7 to volume 4515a9a8-0c73-4773-a967-98fdff017ce5. Nov 10 04:24:00.734801 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Delete backup started, backup: 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7. Nov 10 04:24:00.737856 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Delete started for backup=2dfae63e-fb1b-4968-9870-e7af4b7fa2e7 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:24:00.738281 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Trying diff format basename='volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' for backup base image of volume 6c30d2ca-7042-44c8-b348-092a2c517edb. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:24:00.838104 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Looking for snapshot of backup base 'volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:24:00.839047 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Found snapshot 'backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:24:00.839430 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Deleting backup snapshot='backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7.snap.1636518220.4561949' {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 10 04:24:01.341979 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Deleting backup base image='volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' of volume 6c30d2ca-7042-44c8-b348-092a2c517edb. Nov 10 04:24:02.037024 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Base backup image='volume-6c30d2ca-7042-44c8-b348-092a2c517edb.backup.2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' of volume 6c30d2ca-7042-44c8-b348-092a2c517edb deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:24:02.097379 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Delete of backup '2dfae63e-fb1b-4968-9870-e7af4b7fa2e7' for volume '6c30d2ca-7042-44c8-b348-092a2c517edb' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:24:02.103477 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:24:02.137874 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Created reservations ['3a597dc6-70c9-4678-878d-e8657f1281ef', '26e14960-2c98-4db1-90d9-84976a6d3529'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:24:02.186791 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-239bd664-4cd4-4795-aeb7-2e7a659d7e74 tempest-VolumesBackupsTest-1397900487 None] Delete backup finished, backup 2dfae63e-fb1b-4968-9870-e7af4b7fa2e7 deleted. Nov 10 04:24:26.240937 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:24:26.242935 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:25:26.240427 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:25:26.241489 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:26:26.244343 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:26:26.245545 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:27:02.872000 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Create backup started, backup: c79ad7c7-e609-4ff6-9c72-4daee355890b volume: ef9710e0-7ba9-4d0f-a737-61c1a9dac93f. Nov 10 04:27:02.882270 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:02.884205 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.896733 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:02.897244 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[c00f4bbc-e88c-49e1-97d2-e8fdc0e232de]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:02.900546 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.905039 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[bc751b71-b0c0-4c80-824c-9cd2e4d96c4c]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:02.905651 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:02.906935 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc751b71-b0c0-4c80-824c-9cd2e4d96c4c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:02.909513 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.914880 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[03bbf88c-dc93-4c05-80d1-a8cb9738235d]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:02.915275 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:02.916536 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[03bbf88c-dc93-4c05-80d1-a8cb9738235d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:02.918104 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.928164 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.010s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:02.928794 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4627a49-ac74-475a-8404-cff811b48400]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:02.931541 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.942507 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.012s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:02.942958 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[53546435-d956-4dd5-8b81-54852079d448]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:02.946944 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.952461 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:02.953047 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[c31bf016-c200-4df3-af40-6537c657bf48]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:02.953739 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:02.974782 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:02.976154 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] <== get_connector_properties: return (92ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:03.720366 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:27:03.721792 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ef9710e0-7ba9-4d0f-a737-61c1a9dac93f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:03.723595 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:27:03.773712 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] <== connect_volume: return (51ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:03.781420 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:03.809665 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Starting backup of volume='ef9710e0-7ba9-4d0f-a737-61c1a9dac93f'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:27:03.810544 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Volume file is RBD: attempting optimized backup {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 10 04:27:03.862112 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Creating base image 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' {{(pid=127755) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 10 04:27:03.862542 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:27:03.914699 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Using --from-snap 'None' for incremental backup of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f. {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 10 04:27:03.915834 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Creating backup snapshot='backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434' {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 10 04:27:04.545218 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Performing differential transfer from 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f' to 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:27:04.545948 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_p10ycirx --pool volumes volumes/volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f@backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:27:04.546527 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:27:05.573165 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Differential backup transfer completed in 1.0275s {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 10 04:27:05.573815 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:27:05.588380 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:27:05.588896 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:27:05.595124 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:27:05.595510 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:27:05.601479 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] No metadata type 'volume-glance-metadata' available {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 10 04:27:05.602019 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Backing up metadata for volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:27:05.641636 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Backup 'c79ad7c7-e609-4ff6-9c72-4daee355890b' of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:27:05.643423 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ef9710e0-7ba9-4d0f-a737-61c1a9dac93f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:05.668162 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] <== disconnect_volume: return (23ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:05.727471 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-1a46fe72-9ad8-4ab5-a62f-d4615bff1374 tempest-VolumesBackupsAdminTest-2123084135 None] Create backup finished. backup: c79ad7c7-e609-4ff6-9c72-4daee355890b. Nov 10 04:27:06.172378 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-f30b476e-39ef-493e-9b53-9fae12768020 tempest-VolumesBackupsAdminTest-725506003 None] Export record started, backup: c79ad7c7-e609-4ff6-9c72-4daee355890b. Nov 10 04:27:06.174622 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-f30b476e-39ef-493e-9b53-9fae12768020 tempest-VolumesBackupsAdminTest-725506003 None] Export record finished, backup c79ad7c7-e609-4ff6-9c72-4daee355890b exported. Nov 10 04:27:06.309536 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-ae5e018c-91c1-4b9d-8d14-cf0033a690a5 tempest-VolumesBackupsAdminTest-725506003 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiNTE0ZGYxMDYtYWJjYi00MzRkLWI5OWItYzJjMjNkNTI0MzI5IiwgInVzZXJfaWQiOiAiNDJhMDg4ODQwZTMwNDIyNjgwYTA5NGExMDgyOWJmNGMiLCAicHJvamVjdF9pZCI6ICJlYjJiZWM4OGFjNWE0ZTNmYTE3MjMxNGY3YWZmYzA4NyIsICJ2b2x1bWVfaWQiOiAiZWY5NzEwZTAtN2JhOS00ZDBmLWE3MzctNjFjMWE5ZGFjOTNmIiwgImhvc3QiOiAibmNkdi04MTcxNzYtMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC05MDY1NzgxNTUiLCAiZGlzcGxheV9kZXNjcmlwdGlvbiI6IG51bGwsICJzZXJ2aWNlX21ldGFkYXRhIjogIntcImJhc2VcIjogXCJ2b2x1bWUtZWY5NzEwZTAtN2JhOS00ZDBmLWE3MzctNjFjMWE5ZGFjOTNmLmJhY2t1cC5jNzlhZDdjNy1lNjA5LTRmZjYtOWM3Mi00ZGFlZTM1NTg5MGJcIn0iLCAic2VydmljZSI6ICJjaW5kZXIuYmFja3VwLmRyaXZlcnMuY2VwaC5DZXBoQmFja3VwRHJpdmVyIiwgIm9iamVjdF9jb3VudCI6IDAsICJ0ZW1wX3ZvbHVtZV9pZCI6IG51bGwsICJ0ZW1wX3NuYXBzaG90X2lkIjogbnVsbCwgIm51bV9kZXBlbmRlbnRfYmFja3VwcyI6IDAsICJzbmFwc2hvdF9pZCI6IG51bGwsICJkYXRhX3RpbWVzdGFtcCI6ICIyMDIxLTExLTEwVDA0OjI3OjAzWiIsICJyZXN0b3JlX3ZvbHVtZV9pZCI6IG51bGwsICJtZXRhZGF0YSI6IHt9LCAiZW5jcnlwdGlvbl9rZXlfaWQiOiBudWxsLCAiY3JlYXRlZF9hdCI6ICIyMDIxLTExLTEwVDA0OjI3OjAzWiIsICJ1cGRhdGVkX2F0IjogIjIwMjEtMTEtMTBUMDQ6Mjc6MDZaIiwgImRlbGV0ZWRfYXQiOiBudWxsLCAiZGVsZXRlZCI6IGZhbHNlfQ==. Nov 10 04:27:06.312800 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-ae5e018c-91c1-4b9d-8d14-cf0033a690a5 tempest-VolumesBackupsAdminTest-725506003 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:06.348175 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-ae5e018c-91c1-4b9d-8d14-cf0033a690a5 tempest-VolumesBackupsAdminTest-725506003 None] Import record id 514df106-abcb-434d-b99b-c2c23d524329 metadata from driver finished. Nov 10 04:27:08.660109 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Restore backup started, backup: c79ad7c7-e609-4ff6-9c72-4daee355890b volume: a78d6171-02e2-46a0-a1bd-3f990aa2e532. Nov 10 04:27:08.670814 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:08.674835 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.685996 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:08.686341 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4ae6eb8-2496-40a0-bdbd-83f280d85ca7]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:08.687959 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[387bf7ef-f4c9-4f9b-a55d-e8e330a2cabf]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:08.696570 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[387bf7ef-f4c9-4f9b-a55d-e8e330a2cabf]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:08.697817 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a29ec6d5-5c12-42a2-8b2a-7ab8b06ac67a]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:08.704653 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:08.705738 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[a29ec6d5-5c12-42a2-8b2a-7ab8b06ac67a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:08.706556 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.715382 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:08.716039 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[78e13c1e-7220-4994-b483-acff4531ce9d]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:08.717517 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.725791 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:08.726106 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[c54785f8-be50-41c1-b6f4-2c01b4e04e75]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:08.729200 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.737243 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:08.737579 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed3f369f-d301-4080-a1c1-3f3496a727d2]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:08.738908 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:08.758468 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:08.759544 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] <== get_connector_properties: return (88ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:09.439649 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-2d587237-930c-422a-8d75-89b95d593445 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:09.454496 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:27:09.455930 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-a78d6171-02e2-46a0-a1bd-3f990aa2e532', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a78d6171-02e2-46a0-a1bd-3f990aa2e532', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:09.457566 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:27:09.499101 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] <== connect_volume: return (42ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:09.513584 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Starting restore from Ceph backup=c79ad7c7-e609-4ff6-9c72-4daee355890b to volume=volume-a78d6171-02e2-46a0-a1bd-3f990aa2e532 {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 10 04:27:09.577931 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Looking for snapshot of backup base 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:27:09.578351 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Found snapshot 'backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:27:09.583582 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Volume file is RBD. {{(pid=127755) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 10 04:27:09.619158 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Attempting differential restore. {{(pid=127755) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 10 04:27:09.619636 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Attempting incremental restore from base='volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' snap='backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434' {{(pid=127755) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 10 04:27:09.620024 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Performing differential transfer from 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' to 'volume-a78d6171-02e2-46a0-a1bd-3f990aa2e532' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:27:09.620412 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b@backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:27:09.620809 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_7kq7cgee --pool volumes - volumes/volume-a78d6171-02e2-46a0-a1bd-3f990aa2e532' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:27:10.824548 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Restore transfer completed in 1.2039s {{(pid=127755) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 10 04:27:10.829969 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 10 04:27:10.852110 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Restoring volume base metadata {{(pid=127755) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 10 04:27:10.873002 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] No metadata of type 'volume-metadata' to restore {{(pid=127755) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 10 04:27:10.873379 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=127755) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 10 04:27:10.876705 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Restore to volume a78d6171-02e2-46a0-a1bd-3f990aa2e532 finished successfully. {{(pid=127755) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 10 04:27:10.878272 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-a78d6171-02e2-46a0-a1bd-3f990aa2e532', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a78d6171-02e2-46a0-a1bd-3f990aa2e532', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:10.888308 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] <== disconnect_volume: return (8ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:10.932046 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.manager [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Encryption key ID for volume a78d6171-02e2-46a0-a1bd-3f990aa2e532 already matches encryption key ID in backup c79ad7c7-e609-4ff6-9c72-4daee355890b. {{(pid=127755) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 10 04:27:10.962333 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-f7005fec-5678-40c5-ac65-db427cb5b6be tempest-VolumesBackupsAdminTest-2123084135 None] Finished restoring backup c79ad7c7-e609-4ff6-9c72-4daee355890b to volume a78d6171-02e2-46a0-a1bd-3f990aa2e532. Nov 10 04:27:11.977127 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Delete backup started, backup: 514df106-abcb-434d-b99b-c2c23d524329. Nov 10 04:27:11.979759 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Delete started for backup=514df106-abcb-434d-b99b-c2c23d524329 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:27:11.980251 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Trying diff format basename='volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' for backup base image of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:27:12.032193 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Looking for snapshot of backup base 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:27:12.033022 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Backup '514df106-abcb-434d-b99b-c2c23d524329' has no snapshot {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}} Nov 10 04:27:12.033399 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] No backup snapshot to delete {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Nov 10 04:27:12.042443 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Backup base image of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f still has 1 snapshots so skipping base image delete. Nov 10 04:27:12.086356 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Metadata backup object 'backup.514df106-abcb-434d-b99b-c2c23d524329.meta' not found - ignoring {{(pid=127755) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}} Nov 10 04:27:12.089723 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Delete of backup '514df106-abcb-434d-b99b-c2c23d524329' for volume 'ef9710e0-7ba9-4d0f-a737-61c1a9dac93f' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:27:12.096028 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:12.119210 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Created reservations ['f83a5cdb-4f87-461e-9bae-1ea0dcc7c28b', '519c1473-50ba-4d24-9ce8-eb2588d9cfe7'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:27:12.143734 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-1c99c3be-442f-4133-bbd7-1bc15b443b84 tempest-VolumesBackupsAdminTest-725506003 None] Delete backup finished, backup 514df106-abcb-434d-b99b-c2c23d524329 deleted. Nov 10 04:27:13.060926 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Delete backup started, backup: c79ad7c7-e609-4ff6-9c72-4daee355890b. Nov 10 04:27:13.060926 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Delete started for backup=c79ad7c7-e609-4ff6-9c72-4daee355890b {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:27:13.060926 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Trying diff format basename='volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' for backup base image of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:27:13.104893 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Looking for snapshot of backup base 'volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:27:13.105299 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Found snapshot 'backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:27:13.105675 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Deleting backup snapshot='backup.c79ad7c7-e609-4ff6-9c72-4daee355890b.snap.1636518423.9154434' {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 10 04:27:13.798889 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Deleting backup base image='volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f. Nov 10 04:27:14.032821 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Base backup image='volume-ef9710e0-7ba9-4d0f-a737-61c1a9dac93f.backup.c79ad7c7-e609-4ff6-9c72-4daee355890b' of volume ef9710e0-7ba9-4d0f-a737-61c1a9dac93f deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:27:14.083413 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Delete of backup 'c79ad7c7-e609-4ff6-9c72-4daee355890b' for volume 'ef9710e0-7ba9-4d0f-a737-61c1a9dac93f' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:27:14.089566 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:14.121292 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Created reservations ['fe646693-75d1-4ee3-885b-a853bb588057', '7d7614eb-8749-42ef-9e85-edfec4dced26'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:27:14.144771 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-0e4f161b-1cf4-4081-a7a4-d89c20b329fe tempest-VolumesBackupsAdminTest-2123084135 None] Delete backup finished, backup c79ad7c7-e609-4ff6-9c72-4daee355890b deleted. Nov 10 04:27:16.556320 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Create backup started, backup: 4e8657d8-7e16-4758-94f4-8b5335a9cc69 volume: 2326d381-fd5e-45e7-9597-287caa95ec86. Nov 10 04:27:16.564981 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:16.566368 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.573275 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:16.573825 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9dd6ade-e9c8-48e1-a758-cf73c023bda5]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:16.575201 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.580821 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4f9ba9e7-11d5-491c-82fb-7aea858e5b8c]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:16.581417 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:16.582563 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f9ba9e7-11d5-491c-82fb-7aea858e5b8c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:16.583148 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.589256 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[acc949b6-767e-49e9-9848-56486081c9e5]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:16.589818 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:16.590953 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[acc949b6-767e-49e9-9848-56486081c9e5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:16.593288 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.599969 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:16.600435 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd1bf9af-51f4-4315-beb6-67944a38dbe0]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:16.601836 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.610349 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:16.610663 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b71550f-4955-4851-8476-d4b28e2870dd]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:16.611964 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.619285 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:16.619797 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8115324-c9b0-48ad-a500-01deec1d54c4]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:16.620819 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:16.639736 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:16.640926 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] <== get_connector_properties: return (75ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:17.422927 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:27:17.424329 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-2326d381-fd5e-45e7-9597-287caa95ec86', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '2326d381-fd5e-45e7-9597-287caa95ec86', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:17.425493 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:27:17.464530 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] <== connect_volume: return (39ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:17.471837 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:17.491556 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Starting backup of volume='2326d381-fd5e-45e7-9597-287caa95ec86'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:27:17.492472 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Volume file is RBD: attempting optimized backup {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 10 04:27:17.515539 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Creating base image 'volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' {{(pid=127755) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 10 04:27:17.515940 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:27:17.545607 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Using --from-snap 'None' for incremental backup of volume 2326d381-fd5e-45e7-9597-287caa95ec86. {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 10 04:27:17.546009 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Creating backup snapshot='backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69.snap.1636518437.5456276' {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 10 04:27:18.531765 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Performing differential transfer from 'volume-2326d381-fd5e-45e7-9597-287caa95ec86' to 'volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:27:18.532413 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd__ktgfdsi --pool volumes volumes/volume-2326d381-fd5e-45e7-9597-287caa95ec86@backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69.snap.1636518437.5456276 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:27:18.532774 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:27:19.559443 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Differential backup transfer completed in 1.0274s {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 10 04:27:19.560068 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:27:19.573015 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:27:19.573420 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:27:19.579976 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:27:19.580353 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:27:19.586604 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] No metadata type 'volume-glance-metadata' available {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 10 04:27:19.587143 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Backing up metadata for volume 2326d381-fd5e-45e7-9597-287caa95ec86. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:27:19.636697 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Backup '4e8657d8-7e16-4758-94f4-8b5335a9cc69' of volume 2326d381-fd5e-45e7-9597-287caa95ec86 finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:27:19.638265 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-2326d381-fd5e-45e7-9597-287caa95ec86', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '2326d381-fd5e-45e7-9597-287caa95ec86', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:19.659999 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] <== disconnect_volume: return (20ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:19.717849 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-93092fd1-e003-4fad-a8b8-6ac0edb39d83 tempest-VolumesBackupsAdminTest-2123084135 None] Create backup finished. backup: 4e8657d8-7e16-4758-94f4-8b5335a9cc69. Nov 10 04:27:20.739042 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-54b24d48-849e-4826-b656-fcb6faaa891f tempest-VolumesBackupsAdminTest-725506003 None] Reset backup status started, backup_id: 4e8657d8-7e16-4758-94f4-8b5335a9cc69, status: error. Nov 10 04:27:20.739042 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-54b24d48-849e-4826-b656-fcb6faaa891f tempest-VolumesBackupsAdminTest-725506003 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver. Nov 10 04:27:20.810628 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Delete backup started, backup: 4e8657d8-7e16-4758-94f4-8b5335a9cc69. Nov 10 04:27:20.814028 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Delete started for backup=4e8657d8-7e16-4758-94f4-8b5335a9cc69 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:27:20.814453 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Trying diff format basename='volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' for backup base image of volume 2326d381-fd5e-45e7-9597-287caa95ec86. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:27:20.873532 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Looking for snapshot of backup base 'volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:27:20.873916 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Found snapshot 'backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69.snap.1636518437.5456276' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:27:20.874282 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Deleting backup snapshot='backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69.snap.1636518437.5456276' {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 10 04:27:21.600034 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Deleting backup base image='volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' of volume 2326d381-fd5e-45e7-9597-287caa95ec86. Nov 10 04:27:21.781529 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Base backup image='volume-2326d381-fd5e-45e7-9597-287caa95ec86.backup.4e8657d8-7e16-4758-94f4-8b5335a9cc69' of volume 2326d381-fd5e-45e7-9597-287caa95ec86 deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:27:21.819078 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Delete of backup '4e8657d8-7e16-4758-94f4-8b5335a9cc69' for volume '2326d381-fd5e-45e7-9597-287caa95ec86' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:27:21.824698 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:21.849110 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Created reservations ['62cb31c6-c31d-4110-8d9b-fc6558c7ba07', 'bf9eebe9-6d43-49c1-b76d-21a8f3be40f6'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:27:21.874057 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-fd77f581-8321-4df6-80b3-3fa3836d95d0 tempest-VolumesBackupsAdminTest-2123084135 None] Delete backup finished, backup 4e8657d8-7e16-4758-94f4-8b5335a9cc69 deleted. Nov 10 04:27:22.711559 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Create backup started, backup: 358aee37-6277-401c-8a64-5776959f9c03 volume: 086f3e54-f747-446a-929a-167b2a96c199. Nov 10 04:27:22.719321 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.13', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:22.720746 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.727945 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:22.728560 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2f2ac2d-f2a8-4b77-817e-3060eace7028]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:332b4164e5b9\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:22.729896 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.735185 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[7062ae40-525f-47a8-abc5-18a951ab9b0c]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:22.735785 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:22.736919 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[7062ae40-525f-47a8-abc5-18a951ab9b0c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:22.737285 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.743226 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3245627c-741a-4918-b0e4-62ef4ff7ae78]: [Errno 2] No such file or directory: 'systool' {{(pid=150467) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: Traceback (most recent call last): Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: ret = func(*f_args, **f_kwargs) Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: return func(*args, **kwargs) Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: return putils.execute(on_execute=on_execute, Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: obj = subprocess.Popen(cmd, Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 10 04:27:22.743809 ncdv-817176-1 cinder-backup[127755]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 10 04:27:22.744917 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[3245627c-741a-4918-b0e4-62ef4ff7ae78]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:22.745492 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.753295 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:22.753609 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[6087a472-f100-45f6-91ac-228e8b1e87ca]: (4, ('/dev/sda1\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:22.754732 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.762612 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:22.762921 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[22cea4ef-2753-499a-8466-61ae9586b75c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:22.763964 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.770273 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=150467) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:22.770587 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo.privsep.daemon [-] privsep: reply[6007d6df-f1cd-4771-9bc5-b04854be7782]: (4, ('7c46513b-be29-7d44-8ae5-e61434553583\n', '')) {{(pid=150467) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 10 04:27:22.771304 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Running cmd (subprocess): nvme version {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 10 04:27:22.790519 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_concurrency.processutils [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=127755) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 10 04:27:22.791500 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.utils [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] <== get_connector_properties: return (71ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.13', 'host': 'ncdv-817176-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:332b4164e5b9', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '7c46513b-be29-7d44-8ae5-e61434553583', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:2feed0f8-b7b2-4ba0-be0a-ca24b61b8bf5'} {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:23.615781 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connector [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Factory for rbd on None {{(pid=127755) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 10 04:27:23.617251 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-086f3e54-f747-446a-929a-167b2a96c199', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '086f3e54-f747-446a-929a-167b2a96c199', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:23.618301 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.linuxrbd [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] opening connection to ceph cluster (timeout=-1). {{(pid=127755) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 10 04:27:23.665832 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] <== connect_volume: return (47ms) {'path': } {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:23.675324 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:23.697129 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Starting backup of volume='086f3e54-f747-446a-929a-167b2a96c199'. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 10 04:27:23.698369 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Volume file is RBD: attempting optimized backup {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 10 04:27:23.729639 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Creating base image 'volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' {{(pid=127755) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 10 04:27:23.730048 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=127755) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 10 04:27:23.771206 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Using --from-snap 'None' for incremental backup of volume 086f3e54-f747-446a-929a-167b2a96c199. {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 10 04:27:23.771751 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Creating backup snapshot='backup.358aee37-6277-401c-8a64-5776959f9c03.snap.1636518443.7712288' {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 10 04:27:24.600897 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Performing differential transfer from 'volume-086f3e54-f747-446a-929a-167b2a96c199' to 'volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' {{(pid=127755) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 10 04:27:24.601509 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_b93dhkff --pool volumes volumes/volume-086f3e54-f747-446a-929a-167b2a96c199@backup.358aee37-6277-401c-8a64-5776959f9c03.snap.1636518443.7712288 -' into... {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 10 04:27:24.601917 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' {{(pid=127755) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 10 04:27:25.633223 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Differential backup transfer completed in 1.0318s {{(pid=127755) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 10 04:27:25.633846 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Getting metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 10 04:27:25.648016 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=127755) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 10 04:27:25.648435 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Getting metadata type 'volume-metadata' {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 10 04:27:25.655161 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] No metadata type 'volume-metadata' available {{(pid=127755) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 10 04:27:25.655545 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Getting metadata type 'volume-glance-metadata' {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 10 04:27:25.661611 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.driver [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] No metadata type 'volume-glance-metadata' available {{(pid=127755) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 10 04:27:25.662266 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Backing up metadata for volume 086f3e54-f747-446a-929a-167b2a96c199. {{(pid=127755) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 10 04:27:25.695090 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Backup '358aee37-6277-401c-8a64-5776959f9c03' of volume 086f3e54-f747-446a-929a-167b2a96c199 finished. {{(pid=127755) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 10 04:27:25.696741 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-086f3e54-f747-446a-929a-167b2a96c199', 'hosts': ['10.222.0.13'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '086f3e54-f747-446a-929a-167b2a96c199', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 10 04:27:25.716484 ncdv-817176-1 cinder-backup[127755]: DEBUG os_brick.initiator.connectors.rbd [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] <== disconnect_volume: return (19ms) None {{(pid=127755) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 10 04:27:25.790528 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-4a6bd327-a548-4072-9b75-564b2d36c489 tempest-VolumesBackupsV39Test-739413831 None] Create backup finished. backup: 358aee37-6277-401c-8a64-5776959f9c03. Nov 10 04:27:26.241378 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:27:26.241749 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:27:26.975006 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Delete backup started, backup: 358aee37-6277-401c-8a64-5776959f9c03. Nov 10 04:27:26.977011 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Delete started for backup=358aee37-6277-401c-8a64-5776959f9c03 {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 10 04:27:26.977620 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Trying diff format basename='volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' for backup base image of volume 086f3e54-f747-446a-929a-167b2a96c199. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 10 04:27:27.030541 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Looking for snapshot of backup base 'volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 10 04:27:27.031078 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Found snapshot 'backup.358aee37-6277-401c-8a64-5776959f9c03.snap.1636518443.7712288' {{(pid=127755) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 10 04:27:27.031299 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Deleting backup snapshot='backup.358aee37-6277-401c-8a64-5776959f9c03.snap.1636518443.7712288' {{(pid=127755) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 10 04:27:27.662986 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Deleting backup base image='volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' of volume 086f3e54-f747-446a-929a-167b2a96c199. Nov 10 04:27:27.984563 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Base backup image='volume-086f3e54-f747-446a-929a-167b2a96c199.backup.358aee37-6277-401c-8a64-5776959f9c03' of volume 086f3e54-f747-446a-929a-167b2a96c199 deleted. {{(pid=127755) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 10 04:27:28.037046 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.backup.drivers.ceph [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Delete of backup '358aee37-6277-401c-8a64-5776959f9c03' for volume '086f3e54-f747-446a-929a-167b2a96c199' finished. {{(pid=127755) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 10 04:27:28.043969 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_db.sqlalchemy.engines [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127755) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 10 04:27:28.068391 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.quota [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Created reservations ['f8b3cbb0-d7bf-47da-b31f-f6fb439de9d0', 'bcfc578e-cc7e-4e92-9861-88baa7553544'] {{(pid=127755) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 10 04:27:28.093261 ncdv-817176-1 cinder-backup[127755]: INFO cinder.backup.manager [None req-6f5c65e8-c692-454b-9c08-60f7a8d76da3 tempest-VolumesBackupsV39Test-739413831 None] Delete backup finished, backup 358aee37-6277-401c-8a64-5776959f9c03 deleted. Nov 10 04:28:26.246420 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:28:26.247102 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:29:26.252893 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:29:26.253662 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:30:26.248038 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:30:26.248909 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:31:26.253341 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:31:26.254037 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:32:26.257831 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:32:26.258576 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:33:26.263499 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:33:26.264196 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:34:26.259777 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:34:26.260452 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:35:26.264832 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:35:26.265392 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:36:26.269475 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:36:26.270143 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:37:26.274515 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:37:26.275184 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:38:26.270812 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:38:26.271486 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:39:26.276595 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:39:26.277309 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:40:26.281188 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:40:26.281868 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:41:26.285697 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:41:26.286383 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:42:26.291175 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:42:26.291824 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:43:26.296296 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:43:26.297066 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:44:26.300867 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:44:26.301541 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:45:26.305871 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:45:26.306517 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:46:26.311558 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:46:26.312175 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:47:26.316405 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:47:26.317124 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:48:26.322182 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:48:26.322850 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:50:26.327921 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:50:26.328672 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:51:26.332704 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:51:26.333871 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:52:26.329595 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:52:26.330269 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:53:26.326893 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:53:26.327581 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:54:26.323152 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:54:26.323888 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:55:26.328780 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:55:26.329444 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:56:26.324993 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:56:26.325722 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:57:26.331072 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:57:26.331803 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:58:26.336791 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:58:26.337324 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 04:59:26.332333 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 04:59:26.333060 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:00:26.337472 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:00:26.338132 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:01:26.341878 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:01:26.342584 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:02:26.347425 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:02:26.348113 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:03:26.352399 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:03:26.353149 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:04:26.356734 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:04:26.357498 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:05:26.362101 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:05:26.362865 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:06:26.357941 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:06:26.359204 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:07:26.363692 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:07:26.364389 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:08:26.368306 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:08:26.369083 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:09:26.372983 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:09:26.373688 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:10:26.378394 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:10:26.379115 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:11:26.377028 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:11:26.377710 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:12:26.381752 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:12:26.382480 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:13:26.386528 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:13:26.387234 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:14:26.391115 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:14:26.391884 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:15:26.387397 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:15:26.388143 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:16:26.393714 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:16:26.394400 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:17:26.390762 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:17:26.391480 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:18:26.386346 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:18:26.387064 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:19:26.391806 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:19:26.392538 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:20:26.397306 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:20:26.398008 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:21:26.402341 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:21:26.403051 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:22:26.407993 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:22:26.408697 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:23:26.413212 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:23:26.413927 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 10 05:24:26.409065 ncdv-817176-1 cinder-backup[127755]: DEBUG oslo_service.periodic_task [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=127755) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 10 05:24:26.409758 ncdv-817176-1 cinder-backup[127755]: DEBUG cinder.manager [None req-4fab238e-ee0b-4025-964f-9967ac90c770 None None] Notifying Schedulers of capabilities ... {{(pid=127755) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}