-- Logs begin at Sat 2022-02-05 20:01:18 UTC, end at Sat 2022-02-05 22:19:27 UTC. -- Feb 05 20:37:20.023369 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service. Feb 05 20:37:23.413868 ncdv-master-1 cinder-backup[122151]: INFO cinder.cmd.backup [-] Backup running with 1 processes. Feb 05 20:37:23.414671 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=122151) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Feb 05 20:37:23.415004 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=122151) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Feb 05 20:37:23.589133 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_db.sqlalchemy.engines [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122151) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 20:37:23.833719 ncdv-master-1 cinder-backup[122151]: INFO cinder.rpc [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Feb 05 20:37:23.838160 ncdv-master-1 cinder-backup[122151]: INFO cinder.rpc [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Automatically selected cinder-backup RPC version 2.3 as minimum service version. Feb 05 20:37:23.843044 ncdv-master-1 cinder-backup[122151]: INFO cinder.rpc [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Feb 05 20:37:23.847143 ncdv-master-1 cinder-backup[122151]: INFO cinder.rpc [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Automatically selected cinder-volume RPC version 3.17 as minimum service version. Feb 05 20:37:23.857030 ncdv-master-1 cinder-backup[122151]: INFO cinder.rpc [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Feb 05 20:37:23.862172 ncdv-master-1 cinder-backup[122151]: INFO cinder.rpc [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Feb 05 20:37:23.934494 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_db.sqlalchemy.engines [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122151) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 20:37:23.970259 ncdv-master-1 cinder-backup[122151]: INFO oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Starting 1 workers Feb 05 20:37:23.970259 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Started child 122595 {{(pid=122151) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Feb 05 20:37:23.970732 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Full set of CONF: {{(pid=122151) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Feb 05 20:37:23.971418 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] ******************************************************************************** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Feb 05 20:37:23.971418 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] Configuration options gathered from: {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Feb 05 20:37:23.971746 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Feb 05 20:37:23.972448 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Feb 05 20:37:23.978411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] ================================================================================ {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Feb 05 20:37:23.978411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] allow_availability_zone_fallback = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.978411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] allow_compression_on_image_upload = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.978411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] allowed_direct_url_schemes = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.978411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] api_rate_limit = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.978411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] auth_strategy = keystone {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.979572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_availability_zone = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.979572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_api_class = cinder.backup.api.API {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.979572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_chunk_size = 134217728 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.979572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_conf = /etc/ceph/ceph.conf {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.979572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_image_journals = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.979572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_pool = backups {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.980234 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_stripe_count = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.980234 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_stripe_unit = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.980234 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_ceph_user = cinder-bak {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.980234 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_driver = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.980234 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_driver_init_check_interval = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.981251 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_driver_stats_polling_interval = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.981251 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.981424 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_max_operations = 15 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.988740 ncdv-master-1 cinder-backup[122595]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0) Feb 05 20:37:23.992202 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_metadata_version = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.992659 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_name_template = backup-%s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.993038 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_native_threads_pool_size = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.993422 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_object_number_per_notification = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.993787 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_service_inithost_offload = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.994232 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_timer_interval = 120 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.995153 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_use_same_host = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.997726 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_use_temp_snapshot = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.997726 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backup_workers = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.997726 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] chap_password = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.997726 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] chap_username = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:23.997726 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.002345 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_internal_tenant_project_id = 2f8b5d83bc3c46ed989f17b6ea4e7019 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.002820 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_internal_tenant_user_id = 34ee55021e334a59826d686988c1e2d1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.003283 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] compression_format = gzip {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.003729 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] config_dir = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.004065 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.004487 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] config_source = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.004999 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.005410 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] control_exchange = openstack {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.005837 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] debug = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.006211 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] default_availability_zone = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.011665 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] default_group_type = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.012490 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.014049 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] default_volume_type = ceph {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.014439 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] driver_client_cert = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.014818 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] driver_client_cert_key = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.015183 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] driver_data_namespace = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.015562 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] driver_ssl_cert_path = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.015930 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] driver_ssl_cert_verify = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.016298 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] driver_use_ssl = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.016624 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] enable_new_services = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.017004 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] enable_unsupported_driver = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.017411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] enabled_backends = ['ceph'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.017785 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] enforce_multipath_for_image_xfer = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.018135 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] filter_function = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.018519 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_api_insecure = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.018909 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_api_servers = ['http://10.222.0.32/image'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.019268 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_api_ssl_compression = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.019650 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_ca_certificates_file = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.020065 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_catalog_info = image:glance:publicURL {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.020413 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.020904 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.021281 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.029208 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_num_retries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.029636 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] glance_request_timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.030003 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] goodness_function = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.030595 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] graceful_shutdown_timeout = 5 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.031015 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] group_api_class = cinder.group.api.API {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.031403 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] host = ncdv-master-1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.031803 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iet_conf = /etc/iet/ietd.conf {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.032212 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_compress_on_upload = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.034596 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_conversion_address_space_limit = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.034966 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_conversion_cpu_limit = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.035510 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.038505 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_upload_use_cinder_backend = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.038963 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_upload_use_internal_tenant = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.039359 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_volume_cache_enabled = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.039731 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_volume_cache_max_count = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.040094 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] image_volume_cache_max_size_gb = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.040472 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] instance_format = [instance: %(uuid)s] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.040866 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.041866 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iscsi_iotype = fileio {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.042255 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iscsi_secondary_ip_addresses = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.057832 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iscsi_target_flags = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.058563 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iscsi_write_cache = on {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.059122 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iser_helper = tgtadm {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.059916 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iser_ip_address = 10.222.0.32 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.061098 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iser_port = 3260 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.061721 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.lockutils [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] Lock "cleanup_incomplete_backups_122151" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.004s {{(pid=122595) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Feb 05 20:37:24.062296 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.062985 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] Cleaning up incomplete backup operations. Feb 05 20:37:24.063572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_config_append = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.064735 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.073991 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_dir = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.074691 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_file = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.075204 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_options = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.075774 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_rotate_interval = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.076365 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_rotate_interval_type = days {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.077044 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] log_rotation_type = none {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.077542 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.078028 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.078683 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.079294 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.079808 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.080422 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] max_age = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.080999 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] max_logfile_count = 30 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.082535 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] max_logfile_size_mb = 200 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.083861 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] max_over_subscription_ratio = 20.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.084494 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] message_reap_interval = 86400 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.085087 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] message_ttl = 2592000 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.085683 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] monkey_patch = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.086291 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] monkey_patch_modules = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.086834 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] my_ip = 10.222.0.32 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.087391 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] no_snapshot_gb_quota = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.088567 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] num_iser_scan_tries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.089192 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] num_shell_tries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.089812 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] num_volume_device_scan_tries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.090424 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] nvmet_ns_id = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.091034 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] nvmet_port_id = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.091636 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_max_limit = 1000 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.092217 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_volume_ext_list = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.092866 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.093570 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_volume_listen = 0.0.0.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.094150 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_volume_listen_port = 8776 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.094736 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_volume_use_ssl = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.095317 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] osapi_volume_workers = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.095914 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] per_volume_size_limit = -1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.096473 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] periodic_fuzzy_delay = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.097064 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] periodic_interval = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.097795 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] public_endpoint = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.098370 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] publish_errors = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.098959 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_backup_gigabytes = 1000 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.099598 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_backups = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.100177 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_consistencygroups = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.100766 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.101332 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_gigabytes = 1000 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.101918 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_groups = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.102550 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_snapshots = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.103128 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] quota_volumes = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.104482 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] rate_limit_burst = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.105064 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] rate_limit_except_level = CRITICAL {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.105644 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] rate_limit_interval = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.106246 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] replication_device = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.106839 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] report_discard_supported = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.107421 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] report_interval = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.108092 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] reservation_clean_interval = 86400 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.108610 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] reservation_expire = 86400 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.109201 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] reserved_percentage = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.109845 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.110442 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] restore_discard_excess_bytes = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.111080 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.111791 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] rpc_response_timeout = 120 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.112351 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] run_external_periodic_tasks = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.112953 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.113673 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] scst_target_driver = iscsi {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.114229 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] scst_target_iqn_name = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.114766 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_down_time = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.115395 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] snapshot_name_template = snapshot-%s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.115968 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] split_loggers = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.116779 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] state_path = /opt/stack/data/cinder {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.117361 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] storage_availability_zone = nova {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.117966 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] storage_protocol = iscsi {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.118603 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] syslog_log_facility = LOG_USER {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.119215 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] target_helper = lioadm {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.119903 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] target_ip_address = 10.222.0.32 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.120562 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] target_port = 3260 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.121154 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.121745 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] target_protocol = iscsi {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.122323 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] trace_flags = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.122932 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] transfer_api_class = cinder.transfer.api.API {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.123545 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] transport_url = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.124199 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] until_refresh = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.124764 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_chap_auth = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.125329 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_default_quota_class = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.126027 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_eventlog = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.126630 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_journal = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.127196 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_json = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.127749 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_multipath_for_image_xfer = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.128313 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_stderr = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.128866 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] use_syslog = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.129492 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] verify_glance_signatures = enabled {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.130131 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_api_class = cinder.volume.api.API {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.130888 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_backend_name = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.131464 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_clear = zero {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.132027 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_clear_ionice = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.132608 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_clear_size = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.133178 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.133735 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_copy_bps_limit = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.134412 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_dd_blocksize = 1M {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.135198 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.135821 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_name_template = volume-%s {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.136401 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volume_usage_audit_period = month {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.137061 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.137618 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] watch_log_file = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Feb 05 20:37:24.138335 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.138978 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.139610 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.enforce_new_defaults = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.140178 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.enforce_scope = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.140829 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.policy_default_rule = default {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.141495 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.142117 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.policy_file = policy.yaml {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.142827 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.143424 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.144017 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.144680 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.145261 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.146023 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_concurrency.disable_process_locking = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.146651 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.147292 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.admin_password = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.147876 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.148719 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.admin_token = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.149588 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.admin_user = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.151019 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_admin_prefix = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.151636 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.152230 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_port = 35357 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.152835 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_protocol = https {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.153468 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_section = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.154108 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_type = password {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.154819 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_uri = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.155388 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.auth_version = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.155969 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.cache = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.156551 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.157127 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.157712 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.delay_auth_decision = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.158342 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.158939 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.http_connect_timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.159512 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.160088 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.identity_uri = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.160658 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.include_service_catalog = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.161264 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.insecure = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.161982 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.interface = public {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.162583 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.163179 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.163762 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.164338 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.165037 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.165604 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.167067 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.167703 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.168282 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.168924 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.169486 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.region_name = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.170078 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.170754 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.service_token_roles_required = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.171314 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.service_type = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.171888 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.token_cache_time = 300 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.172619 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.173314 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.backend = dogpile.cache.null {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.173914 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.backend_argument = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.174548 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.config_prefix = cache.oslo {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.175265 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.dead_timeout = 60.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.175848 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.debug_cache_backend = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.176409 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.enable_retry_client = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.176975 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.enable_socket_keepalive = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.177545 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.enabled = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.178126 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.expiration_time = 600 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.178736 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.hashclient_retry_attempts = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.179358 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.hashclient_retry_delay = 1.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.179939 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_dead_retry = 300 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.180628 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.181189 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.181840 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_pool_maxsize = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.182433 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.183066 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.183655 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.memcache_socket_timeout = 1.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.184250 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.proxies = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.184891 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.retry_attempts = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.185534 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.retry_delay = 0.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.186226 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.socket_keepalive_count = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.186827 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.socket_keepalive_idle = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.187411 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.socket_keepalive_interval = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.187997 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.tls_allowed_ciphers = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.188572 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.tls_cafile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.189133 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.tls_certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.189692 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.tls_enabled = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.190264 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cache.tls_keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.190905 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] coordination.backend_url = etcd3+http://10.222.0.32:2379 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.191532 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.backend = sqlalchemy {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.192054 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.connection = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.192672 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.connection_debug = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.193246 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.connection_parameters = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.193827 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.connection_recycle_time = 3600 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.194428 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.connection_trace = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.194992 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.db_inc_retry_interval = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.195636 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.db_max_retries = 20 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.196206 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.db_max_retry_interval = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.196784 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.db_retry_interval = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.197567 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.max_overflow = 50 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.198155 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.max_pool_size = 5 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.199158 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.max_retries = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.199736 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.mysql_enable_ndb = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.200327 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.200898 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.pool_timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.201494 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.retry_interval = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.202078 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.slave_connection = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.202754 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.sqlite_synchronous = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.203328 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] database.use_db_reconnect = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.203954 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.204528 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.205099 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.205779 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.206359 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.207048 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.connection_string = messaging:// {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.207554 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.enabled = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.208141 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.es_doc_type = notification {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.208722 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.es_scroll_size = 10000 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.209311 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.es_scroll_time = 2m {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.209901 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.filter_error_trace = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.210514 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.hmac_keys = SECRET_KEY {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.211098 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.sentinel_service_name = mymaster {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.211773 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.socket_timeout = 0.1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.212335 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] profiler.trace_sqlalchemy = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.213557 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] privsep_osbrick.capabilities = [21] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.214209 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] privsep_osbrick.group = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.214854 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] privsep_osbrick.helper_command = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.215545 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.216144 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.216724 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] privsep_osbrick.user = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.217335 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.217891 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_sys_admin.group = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.218489 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_sys_admin.helper_command = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.219059 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.219627 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.220210 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] cinder_sys_admin.user = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.220797 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.auth_section = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.221373 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.auth_type = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.221939 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.cafile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.222517 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.223105 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.collect_timing = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.223763 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.insecure = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.224330 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.224905 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.send_service_user_token = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.225515 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.split_loggers = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.226164 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] service_user.timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.226813 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.backend_availability_zone = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.227387 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.chap_password = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.228002 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.chap_username = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.229267 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.230070 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.230685 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.231488 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.driver_client_cert = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.232139 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.driver_client_cert_key = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.232700 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.driver_data_namespace = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.233305 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.233897 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.234565 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.driver_use_ssl = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.235147 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.enable_deferred_deletion = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.235714 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.enable_unsupported_driver = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.236372 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.236940 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.filter_function = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.237509 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.goodness_function = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.238090 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.238692 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.239280 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.239857 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.240439 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.241040 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.241618 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iscsi_iotype = fileio {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.242186 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.242788 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iscsi_target_flags = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.243381 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iscsi_write_cache = on {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.243990 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iser_helper = tgtadm {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.244762 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iser_ip_address = 10.222.0.32 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.245337 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iser_port = 3260 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.245938 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.246652 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.247244 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.247838 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.num_shell_tries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.248532 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.249106 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.249676 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.nvmet_port_id = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.250355 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.250992 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rados_connection_interval = 5 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.251557 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rados_connection_retries = 3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.252134 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_ceph_conf = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.252718 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.253285 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.253860 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.254464 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.255050 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_pool = rbd {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.255606 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_secret_uuid = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.256188 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.256853 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.rbd_user = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.257436 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.258000 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.replication_device = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.258608 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.report_discard_supported = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.259198 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.259846 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.reserved_percentage = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.260444 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.scst_target_driver = iscsi {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.261230 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.scst_target_iqn_name = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.261904 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.storage_protocol = iscsi {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.262522 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.target_helper = tgtadm {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.263242 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.target_ip_address = 10.222.0.32 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.263834 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.target_port = 3260 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.264412 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.264997 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.target_protocol = iscsi {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.265568 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.trace_flags = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.266131 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.unique_fqdn_network = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.266846 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.use_chap_auth = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.267412 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.267980 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_backend_name = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.268580 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_clear = zero {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.269159 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_clear_ionice = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.269742 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_clear_size = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.270318 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.271056 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.271771 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.272441 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.273053 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_reports.file_event_handler = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.273644 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.274218 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_reports.log_dir = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.274913 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.276135 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.276866 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.277457 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.278066 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.278660 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.279241 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.279809 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.280398 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.280985 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.281595 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.282182 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.282789 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.283381 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.283978 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.284689 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.285253 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.285839 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.286427 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.287134 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.287732 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.288372 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.ssl = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.288954 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.289531 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.290145 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.290761 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_rabbit.ssl_version = {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.291958 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_notifications.driver = [] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.292541 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_notifications.retry = -1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.293157 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.293729 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] oslo_messaging_notifications.transport_url = **** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.294466 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.295108 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.295704 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.barbican_api_version = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.296273 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.barbican_endpoint = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.296967 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.barbican_endpoint_type = public {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.297682 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.barbican_region_name = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.298251 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.cafile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.298898 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.299479 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.collect_timing = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.300067 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.insecure = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.300642 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.301223 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.number_of_retries = 60 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.301794 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.retry_delay = 1 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.302393 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.send_service_user_token = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.302971 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.split_loggers = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.303538 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.304106 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.verify_ssl = True {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.304663 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican.verify_ssl_path = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.305237 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.auth_section = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.305822 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.auth_type = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.306536 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.cafile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.307281 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.307863 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.collect_timing = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.308429 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.insecure = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.309014 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.309578 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.split_loggers = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.310219 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] barbican_service_user.timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.310852 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.approle_role_id = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.311439 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.approle_secret_id = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.312106 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.cafile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.312676 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.certfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.313256 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.collect_timing = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.313828 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.insecure = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.314413 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.keyfile = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.315016 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.kv_mountpoint = secret {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.315597 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.kv_version = 2 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.316269 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.namespace = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.316881 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.root_token_id = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.317584 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.split_loggers = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.318151 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.ssl_ca_crt_file = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.318794 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.timeout = None {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.319370 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.use_ssl = False {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.319996 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Feb 05 20:37:24.320508 ncdv-master-1 cinder-backup[122151]: DEBUG oslo_service.service [None req-47bd12e6-1a80-4988-8b24-3ce5a1eb9ae1 None None] ******************************************************************************** {{(pid=122151) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Feb 05 20:37:24.345549 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 20:37:24.378920 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.lockutils [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] Lock "cleanup_incomplete_backups_122151" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.318s {{(pid=122595) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Feb 05 20:37:24.386588 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 20:37:24.392358 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:37:24.411157 ncdv-master-1 cinder-backup[122595]: INFO cinder.keymgr.migration [None req-b04066df-d06e-45b5-b5b7-5e68db710af4 None None] Not migrating encryption keys because the ConfKeyManager is still in use. Feb 05 20:37:24.430660 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.service [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] Creating RPC server for service cinder-backup {{(pid=122595) start /opt/stack/cinder/cinder/service.py:231}} Feb 05 20:37:24.439637 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.service [None req-541fd856-7d18-4635-ad11-edaa95418b88 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=122595) start /opt/stack/cinder/cinder/service.py:237}} Feb 05 20:38:31.491311 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:38:31.492090 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:39:31.488735 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:39:31.489422 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:40:31.484314 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:40:31.485000 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:41:31.490089 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:41:31.490898 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:42:31.488361 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:42:31.489032 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:43:31.493157 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:43:31.493856 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:44:31.498597 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:44:31.499164 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:45:31.503436 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:45:31.504115 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:47:31.509553 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:47:31.510260 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:48:31.514960 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:48:31.515602 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:49:31.519654 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:49:31.521757 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:50:31.522551 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:50:31.523577 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:51:31.522231 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:51:31.523336 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:52:31.525254 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:52:31.525885 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:53:31.530286 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:53:31.530926 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:54:31.528357 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:54:31.529559 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:55:31.525233 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:55:31.525233 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:56:31.530973 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:56:31.532250 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:57:31.531852 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:57:31.533396 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:58:31.533957 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:58:31.536259 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 20:59:31.536618 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 20:59:31.537743 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:00:31.535507 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:00:31.536454 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:01:01.908683 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Create backup started, backup: 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 volume: d2ae291c-dfca-46b6-b441-0f74813e2a5c. Feb 05 21:01:01.918105 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:01:01Z,data_timestamp=2022-02-05T21:01:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-691205693',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=77a65ba8-8f0d-4204-9ebf-a6e085ea5d07,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='fd53344e89624261811c75c09ecfabb2',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='a4b7e03fb9fb49f5bd1410ade1eae6e6',volume_id=d2ae291c-dfca-46b6-b441-0f74813e2a5c) Feb 05 21:01:01.979458 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:01.981327 ncdv-master-1 cinder-backup[122595]: INFO oslo.privsep.daemon [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpafndcgxo/privsep.sock'] Feb 05 21:01:02.058600 ncdv-master-1 sudo[144586]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpafndcgxo/privsep.sock Feb 05 21:01:02.061704 ncdv-master-1 sudo[144586]: pam_unix(sudo:session): session opened for user root by (uid=0) Feb 05 21:01:04.692155 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-c02eea2c-8f6b-49d2-a835-a0183021929b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:05.572251 ncdv-master-1 sudo[144586]: pam_unix(sudo:session): session closed for user root Feb 05 21:01:05.578829 ncdv-master-1 cinder-backup[122595]: INFO oslo.privsep.daemon [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Spawned new privsep daemon via rootwrap Feb 05 21:01:05.579414 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Accepted privsep connection to /tmp/tmpafndcgxo/privsep.sock {{(pid=122595) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}} Feb 05 21:01:05.580808 ncdv-master-1 cinder-backup[122595]: INFO oslo.privsep.daemon [-] privsep daemon starting Feb 05 21:01:05.581150 ncdv-master-1 cinder-backup[122595]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Feb 05 21:01:05.581346 ncdv-master-1 cinder-backup[122595]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none Feb 05 21:01:05.581633 ncdv-master-1 cinder-backup[122595]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 144658 Feb 05 21:01:05.587794 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[daa0b3ad-fa64-45a0-9c76-c39556811551]: (2,) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.101997 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.137808 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.035s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:06.138991 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e53ebae-e53f-45e7-845e-a2c7d6b94ff4]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.141956 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.157122 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f288d5be-fe58-456a-889d-3357ec893f06]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:01:06.182288 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:01:06.184382 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[f288d5be-fe58-456a-889d-3357ec893f06]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.187978 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.200176 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[806ae1bc-b796-4a44-b69f-a99ed4c1e1a4]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:01:06.201395 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:01:06.202856 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[806ae1bc-b796-4a44-b69f-a99ed4c1e1a4]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.205253 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.275032 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.070s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:06.276205 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e484c24-dc15-4ebb-9a52-9ef5dcf0c70d]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.277608 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.366825 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.088s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:06.368502 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[f886c5e4-7b5b-46a6-b8d4-ccc17e309718]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.370596 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.402756 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.032s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:06.403534 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[539e0025-1a98-408c-97fc-7bb75f4a73a5]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:06.405343 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:06.480990 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] CMD "nvme version" returned: 0 in 0.076s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:06.487839 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] <== get_connector_properties: return (4506ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:07.827640 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:01:07.830107 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'd2ae291c-dfca-46b6-b441-0f74813e2a5c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:07.831380 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:01:07.887059 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] <== connect_volume: return (55ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:07.912558 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Starting backup of volume='d2ae291c-dfca-46b6-b441-0f74813e2a5c'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:01:07.914162 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Volume file is RBD: attempting optimized backup {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Feb 05 21:01:07.963887 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Creating base image 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' {{(pid=122595) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Feb 05 21:01:07.964612 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:01:09.576565 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Using --from-snap 'None' for incremental backup of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c. {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Feb 05 21:01:09.577851 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Creating backup snapshot='backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437' {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Feb 05 21:01:10.569775 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Performing differential transfer from 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c' to 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:01:10.570798 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_uslurb09 --pool volumes volumes/volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c@backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:01:10.571445 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:01:11.624740 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Differential backup transfer completed in 1.0540s {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Feb 05 21:01:11.626263 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:01:11.647702 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:01:11.648197 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:01:11.666612 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:01:11.667016 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:01:11.688393 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] No metadata type 'volume-glance-metadata' available {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Feb 05 21:01:11.689360 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Backing up metadata for volume d2ae291c-dfca-46b6-b441-0f74813e2a5c. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:01:11.798087 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Backup '77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:01:11.799801 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'd2ae291c-dfca-46b6-b441-0f74813e2a5c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:11.856159 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] <== disconnect_volume: return (55ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:12.026822 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-74b757c0-8546-4072-bef4-051e39dcff55 tempest-VolumesBackupsAdminTest-2077144323 None] Create backup finished. backup: 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07. Feb 05 21:01:12.647703 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-d10631da-d971-4b5c-910b-ed9f47095906 tempest-VolumesBackupsAdminTest-1294663149 None] Export record started, backup: 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07. Feb 05 21:01:12.649722 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-d10631da-d971-4b5c-910b-ed9f47095906 tempest-VolumesBackupsAdminTest-1294663149 None] Export record finished, backup 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 exported. Feb 05 21:01:12.916326 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-93b8aa45-f982-4b3e-a6f2-bc7da1776a25 tempest-VolumesBackupsAdminTest-1294663149 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiZDE2NThhZDgtODkyZi00YjVhLWJlMzUtZjNjZmIxMDRhMzQ5IiwgInVzZXJfaWQiOiAiYTRiN2UwM2ZiOWZiNDlmNWJkMTQxMGFkZTFlYWU2ZTYiLCAicHJvamVjdF9pZCI6ICJmZDUzMzQ0ZTg5NjI0MjYxODExYzc1YzA5ZWNmYWJiMiIsICJ2b2x1bWVfaWQiOiAiZDJhZTI5MWMtZGZjYS00NmI2LWI0NDEtMGY3NDgxM2UyYTVjIiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC02OTEyMDU2OTMiLCAiZGlzcGxheV9kZXNjcmlwdGlvbiI6IG51bGwsICJzZXJ2aWNlX21ldGFkYXRhIjogIntcImJhc2VcIjogXCJ2b2x1bWUtZDJhZTI5MWMtZGZjYS00NmI2LWI0NDEtMGY3NDgxM2UyYTVjLmJhY2t1cC43N2E2NWJhOC04ZjBkLTQyMDQtOWViZi1hNmUwODVlYTVkMDdcIn0iLCAic2VydmljZSI6ICJjaW5kZXIuYmFja3VwLmRyaXZlcnMuY2VwaC5DZXBoQmFja3VwRHJpdmVyIiwgIm9iamVjdF9jb3VudCI6IDAsICJ0ZW1wX3ZvbHVtZV9pZCI6IG51bGwsICJ0ZW1wX3NuYXBzaG90X2lkIjogbnVsbCwgIm51bV9kZXBlbmRlbnRfYmFja3VwcyI6IDAsICJzbmFwc2hvdF9pZCI6IG51bGwsICJkYXRhX3RpbWVzdGFtcCI6ICIyMDIyLTAyLTA1VDIxOjAxOjAyWiIsICJyZXN0b3JlX3ZvbHVtZV9pZCI6IG51bGwsICJtZXRhZGF0YSI6IHt9LCAiZW5jcnlwdGlvbl9rZXlfaWQiOiBudWxsLCAiY3JlYXRlZF9hdCI6ICIyMDIyLTAyLTA1VDIxOjAxOjAyWiIsICJ1cGRhdGVkX2F0IjogIjIwMjItMDItMDVUMjE6MDE6MTJaIiwgImRlbGV0ZWRfYXQiOiBudWxsLCAiZGVsZXRlZCI6IGZhbHNlfQ==. Feb 05 21:01:12.951264 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-93b8aa45-f982-4b3e-a6f2-bc7da1776a25 tempest-VolumesBackupsAdminTest-1294663149 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:13.019605 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-93b8aa45-f982-4b3e-a6f2-bc7da1776a25 tempest-VolumesBackupsAdminTest-1294663149 None] Import record id d1658ad8-892f-4b5a-be35-f3cfb104a349 metadata from driver finished. Feb 05 21:01:15.554442 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Restore backup started, backup: 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 volume: 8ceb9e09-5239-46e7-95b8-cf7d1ee205fe. Feb 05 21:01:15.598156 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:15.603940 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.627926 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.026s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:15.628515 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[50d75589-ab52-4adb-bd27-60aabc9af6d7]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:15.636894 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.653187 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[aec21c0c-9597-488f-b443-b7db46a63c8a]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:01:15.657279 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:01:15.661356 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[aec21c0c-9597-488f-b443-b7db46a63c8a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:15.664379 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.683103 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[940d83ed-3f38-4dd6-bf05-61de06748b46]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:01:15.685818 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:01:15.687426 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[940d83ed-3f38-4dd6-bf05-61de06748b46]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:15.690511 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.720377 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.031s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:15.721354 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[39b63c42-8c5c-41ef-8b6e-3537b8ca2fcd]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:15.723482 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.752976 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.029s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:15.753395 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[324697a0-6bcd-4e51-b460-a72d9603ec0c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:15.754957 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.775381 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.020s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:15.776657 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac9f7fcd-1684-4300-bda6-99b452cbe742]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:15.777747 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:15.840122 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] CMD "nvme version" returned: 0 in 0.062s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:15.842525 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] <== get_connector_properties: return (242ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:16.835802 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:01:16.837396 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-8ceb9e09-5239-46e7-95b8-cf7d1ee205fe', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8ceb9e09-5239-46e7-95b8-cf7d1ee205fe', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:16.838916 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:01:16.937211 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] <== connect_volume: return (98ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:16.958581 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:16.970907 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Starting restore from Ceph backup=77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 to volume=volume-8ceb9e09-5239-46e7-95b8-cf7d1ee205fe {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Feb 05 21:01:17.107491 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Looking for snapshot of backup base 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:01:17.107491 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Found snapshot 'backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:01:17.124627 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Volume file is RBD. {{(pid=122595) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Feb 05 21:01:17.178140 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Attempting differential restore. {{(pid=122595) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Feb 05 21:01:17.179026 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Attempting incremental restore from base='volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' snap='backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437' {{(pid=122595) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Feb 05 21:01:17.179876 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Performing differential transfer from 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' to 'volume-8ceb9e09-5239-46e7-95b8-cf7d1ee205fe' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:01:17.180432 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07@backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:01:17.181165 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_m_26bjdf --pool volumes - volumes/volume-8ceb9e09-5239-46e7-95b8-cf7d1ee205fe' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:01:18.692732 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Restore transfer completed in 1.5126s {{(pid=122595) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Feb 05 21:01:18.693764 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Feb 05 21:01:18.739969 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Restoring volume base metadata {{(pid=122595) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Feb 05 21:01:18.813116 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] No metadata of type 'volume-metadata' to restore {{(pid=122595) put /opt/stack/cinder/cinder/backup/driver.py:345}} Feb 05 21:01:18.814090 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=122595) put /opt/stack/cinder/cinder/backup/driver.py:345}} Feb 05 21:01:18.818281 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Restore to volume 8ceb9e09-5239-46e7-95b8-cf7d1ee205fe finished successfully. {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Feb 05 21:01:18.820095 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-8ceb9e09-5239-46e7-95b8-cf7d1ee205fe', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8ceb9e09-5239-46e7-95b8-cf7d1ee205fe', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': False}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:18.843617 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] <== disconnect_volume: return (22ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:18.982415 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.manager [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Encryption key ID for volume 8ceb9e09-5239-46e7-95b8-cf7d1ee205fe already matches encryption key ID in backup 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07. {{(pid=122595) _run_restore /opt/stack/cinder/cinder/backup/manager.py:824}} Feb 05 21:01:19.102325 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-cfd5b8ac-f5ec-4317-9908-e2f15b115b04 tempest-VolumesBackupsAdminTest-2077144323 None] Finished restoring backup 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 to volume 8ceb9e09-5239-46e7-95b8-cf7d1ee205fe. Feb 05 21:01:20.208791 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Delete backup started, backup: d1658ad8-892f-4b5a-be35-f3cfb104a349. Feb 05 21:01:20.210940 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Delete started for backup=d1658ad8-892f-4b5a-be35-f3cfb104a349 {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:01:20.211372 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Trying diff format basename='volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' for backup base image of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:01:20.354318 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Looking for snapshot of backup base 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:01:20.354833 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Backup 'd1658ad8-892f-4b5a-be35-f3cfb104a349' has no snapshot {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}} Feb 05 21:01:20.355277 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] No backup snapshot to delete {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Feb 05 21:01:20.366845 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Backup base image of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c still has 1 snapshots so skipping base image delete. Feb 05 21:01:20.431543 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Metadata backup object 'backup.d1658ad8-892f-4b5a-be35-f3cfb104a349.meta' not found - ignoring {{(pid=122595) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}} Feb 05 21:01:20.438593 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Delete of backup 'd1658ad8-892f-4b5a-be35-f3cfb104a349' for volume 'd2ae291c-dfca-46b6-b441-0f74813e2a5c' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:01:20.447001 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:20.503170 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Created reservations ['50d489d5-7d5b-40e4-ae4e-3f041432258f', 'd1c06268-38ad-40d9-a884-258c8fc84469'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:01:20.568941 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-0365b177-935c-4bf3-86db-eac58b081628 tempest-VolumesBackupsAdminTest-1294663149 None] Delete backup finished, backup d1658ad8-892f-4b5a-be35-f3cfb104a349 deleted. Feb 05 21:01:21.350555 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Delete backup started, backup: 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07. Feb 05 21:01:21.352985 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Delete started for backup=77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:01:21.353977 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Trying diff format basename='volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' for backup base image of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:01:21.441533 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Looking for snapshot of backup base 'volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:01:21.442075 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Found snapshot 'backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:01:21.442821 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Deleting backup snapshot='backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07.snap.1644094869.577437' {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Feb 05 21:01:21.662925 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Deleting backup base image='volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c. Feb 05 21:01:22.064718 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Base backup image='volume-d2ae291c-dfca-46b6-b441-0f74813e2a5c.backup.77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' of volume d2ae291c-dfca-46b6-b441-0f74813e2a5c deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:01:22.126226 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Delete of backup '77a65ba8-8f0d-4204-9ebf-a6e085ea5d07' for volume 'd2ae291c-dfca-46b6-b441-0f74813e2a5c' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:01:22.132841 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:22.164041 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Created reservations ['d668f164-df5a-47e6-9e11-17200a2460fd', 'e483f471-e7bd-43d9-9bde-953bb8492049'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:01:22.201922 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-4ede1334-94c3-42f9-bb0c-649855a01482 tempest-VolumesBackupsAdminTest-2077144323 None] Delete backup finished, backup 77a65ba8-8f0d-4204-9ebf-a6e085ea5d07 deleted. Feb 05 21:01:24.273429 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Create backup started, backup: 5f2bf439-ecb4-4dad-9c8b-d27d54648d13 volume: c702e501-f624-45c5-8ff4-1f251b54fbee. Feb 05 21:01:24.289549 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:01:24Z,data_timestamp=2022-02-05T21:01:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-854308935',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=5f2bf439-ecb4-4dad-9c8b-d27d54648d13,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='fd53344e89624261811c75c09ecfabb2',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='a4b7e03fb9fb49f5bd1410ade1eae6e6',volume_id=c702e501-f624-45c5-8ff4-1f251b54fbee) Feb 05 21:01:24.350223 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:24.351923 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.363055 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:24.363354 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[a033247b-541e-4287-8ae0-f225283befda]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:24.367318 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.377951 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8d52c4ed-c153-40e7-a889-1873f3130d67]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:01:24.378280 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:01:24.379588 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d52c4ed-c153-40e7-a889-1873f3130d67]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:24.381742 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.389445 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[85549475-14ad-4b2e-9080-951fa2d86a61]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:01:24.389749 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:01:24.391054 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[85549475-14ad-4b2e-9080-951fa2d86a61]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:24.393360 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.407784 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.016s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:24.408334 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[943d06a5-bde0-4388-8aa0-2987a7aeedfd]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:24.411669 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.440332 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.030s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:24.441628 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[9da38b04-39ee-4e03-8ac6-442b1ee1e0dd]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:24.446470 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.461119 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.017s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:24.471607 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[145965fd-b2c4-48f3-9134-528983527057]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:01:24.472380 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:01:24.511013 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] CMD "nvme version" returned: 0 in 0.038s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:01:24.513574 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] <== get_connector_properties: return (161ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:24.685489 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-c02eea2c-8f6b-49d2-a835-a0183021929b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:25.946599 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:01:25.948967 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-c702e501-f624-45c5-8ff4-1f251b54fbee', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'c702e501-f624-45c5-8ff4-1f251b54fbee', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:25.950534 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:01:26.045488 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] <== connect_volume: return (93ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:26.081041 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Starting backup of volume='c702e501-f624-45c5-8ff4-1f251b54fbee'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:01:26.082260 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Volume file is RBD: attempting optimized backup {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Feb 05 21:01:26.127595 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Creating base image 'volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' {{(pid=122595) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Feb 05 21:01:26.128143 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:01:26.180894 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Using --from-snap 'None' for incremental backup of volume c702e501-f624-45c5-8ff4-1f251b54fbee. {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Feb 05 21:01:26.181724 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Creating backup snapshot='backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13.snap.1644094886.1809127' {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Feb 05 21:01:27.166904 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Performing differential transfer from 'volume-c702e501-f624-45c5-8ff4-1f251b54fbee' to 'volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:01:27.166904 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_70y4z79u --pool volumes volumes/volume-c702e501-f624-45c5-8ff4-1f251b54fbee@backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13.snap.1644094886.1809127 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:01:27.166904 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:01:27.602630 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Differential backup transfer completed in 0.4370s {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Feb 05 21:01:27.603089 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:01:27.618522 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:01:27.623258 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:01:27.632202 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:01:27.633458 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:01:27.646213 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] No metadata type 'volume-glance-metadata' available {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Feb 05 21:01:27.647063 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Backing up metadata for volume c702e501-f624-45c5-8ff4-1f251b54fbee. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:01:27.730332 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Backup '5f2bf439-ecb4-4dad-9c8b-d27d54648d13' of volume c702e501-f624-45c5-8ff4-1f251b54fbee finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:01:27.734134 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-c702e501-f624-45c5-8ff4-1f251b54fbee', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'c702e501-f624-45c5-8ff4-1f251b54fbee', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:01:27.771064 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] <== disconnect_volume: return (33ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:01:27.852654 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-4bbbbcb3-b1f5-43f6-8c1c-2205b0fae76d tempest-VolumesBackupsAdminTest-2077144323 None] Create backup finished. backup: 5f2bf439-ecb4-4dad-9c8b-d27d54648d13. Feb 05 21:01:28.614653 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-344dc5fc-e5c3-4cec-aaa7-83f3250b36a5 tempest-VolumesBackupsAdminTest-1294663149 None] Reset backup status started, backup_id: 5f2bf439-ecb4-4dad-9c8b-d27d54648d13, status: error. Feb 05 21:01:28.615951 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-344dc5fc-e5c3-4cec-aaa7-83f3250b36a5 tempest-VolumesBackupsAdminTest-1294663149 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver. Feb 05 21:01:29.819075 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Delete backup started, backup: 5f2bf439-ecb4-4dad-9c8b-d27d54648d13. Feb 05 21:01:29.821207 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Delete started for backup=5f2bf439-ecb4-4dad-9c8b-d27d54648d13 {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:01:29.821635 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Trying diff format basename='volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' for backup base image of volume c702e501-f624-45c5-8ff4-1f251b54fbee. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:01:29.953423 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Looking for snapshot of backup base 'volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:01:29.953820 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Found snapshot 'backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13.snap.1644094886.1809127' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:01:29.954195 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Deleting backup snapshot='backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13.snap.1644094886.1809127' {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Feb 05 21:01:30.256152 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Deleting backup base image='volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' of volume c702e501-f624-45c5-8ff4-1f251b54fbee. Feb 05 21:01:30.736731 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Base backup image='volume-c702e501-f624-45c5-8ff4-1f251b54fbee.backup.5f2bf439-ecb4-4dad-9c8b-d27d54648d13' of volume c702e501-f624-45c5-8ff4-1f251b54fbee deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:01:30.815474 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Delete of backup '5f2bf439-ecb4-4dad-9c8b-d27d54648d13' for volume 'c702e501-f624-45c5-8ff4-1f251b54fbee' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:01:30.822038 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:01:30.873928 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Created reservations ['f259601c-4c06-4484-a195-44d56a0325a5', '8cf2b740-07d9-4092-9a9e-1a5cb274877f'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:01:30.953257 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-c9fb20b1-5e64-4338-a6fe-a49e03bb17cc tempest-VolumesBackupsAdminTest-2077144323 None] Delete backup finished, backup 5f2bf439-ecb4-4dad-9c8b-d27d54648d13 deleted. Feb 05 21:01:31.532141 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:01:31.532729 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:02:31.532860 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:02:31.533994 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:03:31.531845 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:03:31.532467 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:04:31.532560 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:04:31.533303 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:05:31.534574 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:05:31.537451 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:06:31.538338 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:06:31.539685 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:07:31.537584 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:07:31.538208 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:08:31.542491 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:08:31.543205 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:08:49.376063 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Create backup started, backup: f8bc9e59-8980-46a3-98d2-4f280054bdf8 volume: 551c13d3-70f5-4542-af53-7ecb3c3717f7. Feb 05 21:08:49.385151 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:08:49Z,data_timestamp=2022-02-05T21:08:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-919240273',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=f8bc9e59-8980-46a3-98d2-4f280054bdf8,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='0d41cfbe49c14de89e32c8c88afef49a',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='76fea515065c484eb21f2054edc0a299',volume_id=551c13d3-70f5-4542-af53-7ecb3c3717f7) Feb 05 21:08:49.439636 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:08:49.444412 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.455184 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:08:49.455466 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bcdaa15-fda1-47b1-bcbd-c1e7870fc0df]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:08:49.459396 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.467455 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2b7caada-ef72-4c21-802a-c7136b078d27]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:08:49.468004 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:08:49.469259 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b7caada-ef72-4c21-802a-c7136b078d27]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:08:49.472140 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.480247 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3fb93f71-ea69-42eb-aca5-c120c8b8c926]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:08:49.480596 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:08:49.482860 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fb93f71-ea69-42eb-aca5-c120c8b8c926]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:08:49.485720 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.508685 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.025s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:08:49.509108 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[1533f4cd-b723-4190-9415-520c3f295b29]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:08:49.512063 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.545946 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.032s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:08:49.546210 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a67dedf-226f-4689-9215-b33a2b9ff6ed]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:08:49.550196 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.568594 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.020s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:08:49.570836 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b6390ac-dfbf-4c4c-baa7-f65321484f31]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:08:49.573001 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:08:49.610369 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] CMD "nvme version" returned: 0 in 0.037s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:08:49.611601 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] <== get_connector_properties: return (171ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:08:50.596965 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:08:50.598458 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-551c13d3-70f5-4542-af53-7ecb3c3717f7', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '551c13d3-70f5-4542-af53-7ecb3c3717f7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:08:50.599808 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:08:50.659590 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] <== connect_volume: return (60ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:08:50.669045 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:08:50.689533 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Starting backup of volume='551c13d3-70f5-4542-af53-7ecb3c3717f7'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:08:50.690699 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Volume file is RBD: attempting optimized backup {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Feb 05 21:08:50.722220 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Creating base image 'volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' {{(pid=122595) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Feb 05 21:08:50.722691 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:08:50.773741 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Using --from-snap 'None' for incremental backup of volume 551c13d3-70f5-4542-af53-7ecb3c3717f7. {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Feb 05 21:08:50.774587 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Creating backup snapshot='backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8.snap.1644095330.774102' {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Feb 05 21:08:51.580249 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Performing differential transfer from 'volume-551c13d3-70f5-4542-af53-7ecb3c3717f7' to 'volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:08:51.580699 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_rnd98dqv --pool volumes volumes/volume-551c13d3-70f5-4542-af53-7ecb3c3717f7@backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8.snap.1644095330.774102 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:08:51.581179 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:08:52.652451 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Differential backup transfer completed in 1.0709s {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Feb 05 21:08:52.654060 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:08:52.691271 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:08:52.691730 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:08:52.703583 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:08:52.704782 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:08:52.713509 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] No metadata type 'volume-glance-metadata' available {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Feb 05 21:08:52.714298 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Backing up metadata for volume 551c13d3-70f5-4542-af53-7ecb3c3717f7. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:08:52.789768 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Backup 'f8bc9e59-8980-46a3-98d2-4f280054bdf8' of volume 551c13d3-70f5-4542-af53-7ecb3c3717f7 finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:08:52.791691 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-551c13d3-70f5-4542-af53-7ecb3c3717f7', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '551c13d3-70f5-4542-af53-7ecb3c3717f7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:08:52.812229 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] <== disconnect_volume: return (19ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:08:52.884570 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-cd87f56a-8c9f-4a8c-9a38-5e6df7970a9e tempest-VolumesBackupsTest-350919582 None] Create backup finished. backup: f8bc9e59-8980-46a3-98d2-4f280054bdf8. Feb 05 21:08:53.715382 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Delete backup started, backup: f8bc9e59-8980-46a3-98d2-4f280054bdf8. Feb 05 21:08:53.718644 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Delete started for backup=f8bc9e59-8980-46a3-98d2-4f280054bdf8 {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:08:53.719591 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Trying diff format basename='volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' for backup base image of volume 551c13d3-70f5-4542-af53-7ecb3c3717f7. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:08:53.814866 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Looking for snapshot of backup base 'volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:08:53.815346 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Found snapshot 'backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8.snap.1644095330.774102' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:08:53.815825 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Deleting backup snapshot='backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8.snap.1644095330.774102' {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Feb 05 21:08:54.672901 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Deleting backup base image='volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' of volume 551c13d3-70f5-4542-af53-7ecb3c3717f7. Feb 05 21:08:54.719534 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-c02eea2c-8f6b-49d2-a835-a0183021929b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:08:54.946640 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Base backup image='volume-551c13d3-70f5-4542-af53-7ecb3c3717f7.backup.f8bc9e59-8980-46a3-98d2-4f280054bdf8' of volume 551c13d3-70f5-4542-af53-7ecb3c3717f7 deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:08:55.021498 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Delete of backup 'f8bc9e59-8980-46a3-98d2-4f280054bdf8' for volume '551c13d3-70f5-4542-af53-7ecb3c3717f7' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:08:55.048584 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Created reservations ['606a7739-ae87-4a5a-ac41-70f04c066c88', 'cf00a5f7-6a76-49e6-b62a-863ec8b72d53'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:08:55.087994 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-82ec63b1-e341-4d42-b1dd-f00ef7040f2b tempest-VolumesBackupsTest-350919582 None] Delete backup finished, backup f8bc9e59-8980-46a3-98d2-4f280054bdf8 deleted. Feb 05 21:09:03.681110 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Create backup started, backup: cbe785d6-2f9f-4944-9699-f1fe9c4db8ca volume: 607a91f1-1cda-40ed-933c-a0b4f2ed309b. Feb 05 21:09:03.691495 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:09:03Z,data_timestamp=2022-02-05T21:09:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-126632598',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=cbe785d6-2f9f-4944-9699-f1fe9c4db8ca,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='047f2a78ff8f4c988bdcbeb491ca38d8',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='1f2a32b179af423ab2215af2a0cf3d69',volume_id=607a91f1-1cda-40ed-933c-a0b4f2ed309b) Feb 05 21:09:04.608561 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:04.610656 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.630681 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:04.631419 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f684605-896d-49d8-aa82-4516d8c0ae9f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:04.633028 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.642495 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ac68425e-c2d3-4938-8c75-ec568449138f]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:04.643817 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:04.645691 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac68425e-c2d3-4938-8c75-ec568449138f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:04.647159 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.656146 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b1dae983-02b3-4b9f-9060-6d1686d588e7]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:04.656767 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:04.658368 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1dae983-02b3-4b9f-9060-6d1686d588e7]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:04.660182 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.682329 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.020s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:04.683522 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f13b171-ff52-488e-86ab-246413d9470d]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:04.688474 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.716273 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-c02eea2c-8f6b-49d2-a835-a0183021929b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:09:04.721668 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.019s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:04.722955 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c9d4401-4e03-4bda-9a4e-733f1d21c388]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:04.734340 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.749912 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.018s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:04.763460 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[9154b475-fa67-453b-9583-48a6b5503d7d]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:04.768856 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:04.813676 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] CMD "nvme version" returned: 0 in 0.047s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:04.815172 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] <== get_connector_properties: return (205ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:05.438441 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Create backup started, backup: 39685a0c-8fc6-4d89-ac0b-44ed5da25dab volume: 10f43b5e-5fc3-4f9e-be50-14c09c91d474. Feb 05 21:09:05.449250 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:09:05Z,data_timestamp=2022-02-05T21:09:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-313873250',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=39685a0c-8fc6-4d89-ac0b-44ed5da25dab,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='0d41cfbe49c14de89e32c8c88afef49a',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='76fea515065c484eb21f2054edc0a299',volume_id=10f43b5e-5fc3-4f9e-be50-14c09c91d474) Feb 05 21:09:05.542284 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:05.551268 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.570869 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.026s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:05.571236 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[8809f306-3c10-492d-91a6-61f7f7afcdd6]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:05.572620 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.582749 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[93cc6a78-a60d-4a16-865f-70ddac4236f1]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:05.583473 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:05.585063 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[93cc6a78-a60d-4a16-865f-70ddac4236f1]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:05.589109 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.596377 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[af40cca0-ccb6-4dce-b9d0-3c06edf6c628]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:05.596951 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:05.599007 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[af40cca0-ccb6-4dce-b9d0-3c06edf6c628]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:05.603012 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.613031 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.012s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:05.613480 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb34fe7d-6604-4f7a-aeb8-baba9a1117a9]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:05.616779 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.626242 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.011s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:05.626700 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbcb296e-46b2-4dd2-86d4-270dbb938acb]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:05.627991 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.638612 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:05.639266 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[57e7061d-f108-464e-b605-eba273ce6e26]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:05.640406 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:05.689816 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] CMD "nvme version" returned: 0 in 0.049s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:05.691061 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] <== get_connector_properties: return (147ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:05.791362 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:09:05.792834 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '607a91f1-1cda-40ed-933c-a0b4f2ed309b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:05.795803 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:09:05.882768 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] <== connect_volume: return (86ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:05.899081 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:09:05.920569 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Starting backup of volume='607a91f1-1cda-40ed-933c-a0b4f2ed309b'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:09:05.922500 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Volume file is RBD: attempting optimized backup {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Feb 05 21:09:05.995594 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Creating base image 'volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' {{(pid=122595) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Feb 05 21:09:05.996226 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:09:06.056373 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Using --from-snap 'None' for incremental backup of volume 607a91f1-1cda-40ed-933c-a0b4f2ed309b. {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Feb 05 21:09:06.057227 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Creating backup snapshot='backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca.snap.1644095346.0563948' {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Feb 05 21:09:06.591650 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:09:06.593030 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '10f43b5e-5fc3-4f9e-be50-14c09c91d474', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:06.593854 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:09:06.687498 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] <== connect_volume: return (94ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:06.692171 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Performing differential transfer from 'volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b' to 'volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:09:06.692780 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_ezd4yi3u --pool volumes volumes/volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b@backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca.snap.1644095346.0563948 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:09:06.692939 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:09:06.754771 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Starting backup of volume='10f43b5e-5fc3-4f9e-be50-14c09c91d474'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:09:06.755891 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Volume file is RBD: attempting optimized backup {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Feb 05 21:09:06.815645 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Creating base image 'volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' {{(pid=122595) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Feb 05 21:09:06.816445 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:09:06.883220 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Using --from-snap 'None' for incremental backup of volume 10f43b5e-5fc3-4f9e-be50-14c09c91d474. {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Feb 05 21:09:06.884275 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Creating backup snapshot='backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711' {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Feb 05 21:09:07.627608 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Performing differential transfer from 'volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474' to 'volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:09:07.627608 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_o1o47wq5 --pool volumes volumes/volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474@backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:09:07.627608 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:09:07.687180 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Differential backup transfer completed in 0.9943s {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Feb 05 21:09:07.687992 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:09:07.728810 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:09:07.731701 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:09:07.750027 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:09:07.751861 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:09:07.767682 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] No metadata type 'volume-glance-metadata' available {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Feb 05 21:09:07.768243 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Backing up metadata for volume 607a91f1-1cda-40ed-933c-a0b4f2ed309b. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:09:07.859266 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Backup 'cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' of volume 607a91f1-1cda-40ed-933c-a0b4f2ed309b finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:09:07.861227 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '607a91f1-1cda-40ed-933c-a0b4f2ed309b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:07.905621 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] <== disconnect_volume: return (43ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:08.004605 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-e5a9f4cf-e2d7-4b13-9fb4-af2c596ade84 tempest-VolumesBackupsV39Test-1477652955 None] Create backup finished. backup: cbe785d6-2f9f-4944-9699-f1fe9c4db8ca. Feb 05 21:09:09.186770 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Delete backup started, backup: cbe785d6-2f9f-4944-9699-f1fe9c4db8ca. Feb 05 21:09:09.189151 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Delete started for backup=cbe785d6-2f9f-4944-9699-f1fe9c4db8ca {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:09:09.189780 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Trying diff format basename='volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' for backup base image of volume 607a91f1-1cda-40ed-933c-a0b4f2ed309b. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:09:09.485156 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Looking for snapshot of backup base 'volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:09:09.485443 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Found snapshot 'backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca.snap.1644095346.0563948' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:09:09.486049 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Deleting backup snapshot='backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca.snap.1644095346.0563948' {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Feb 05 21:09:11.052387 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Deleting backup base image='volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' of volume 607a91f1-1cda-40ed-933c-a0b4f2ed309b. Feb 05 21:09:12.641679 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Base backup image='volume-607a91f1-1cda-40ed-933c-a0b4f2ed309b.backup.cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' of volume 607a91f1-1cda-40ed-933c-a0b4f2ed309b deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:09:12.734453 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Delete of backup 'cbe785d6-2f9f-4944-9699-f1fe9c4db8ca' for volume '607a91f1-1cda-40ed-933c-a0b4f2ed309b' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:09:12.743333 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:09:12.779786 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Created reservations ['a73dd5a9-e178-4a9b-9ca6-eab0c33e7fa6', 'c50fa11e-96e3-432a-8c33-6f276f892d30'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:09:12.833544 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-d84bf10d-00ae-43e1-b434-b96ba59d1a27 tempest-VolumesBackupsV39Test-1477652955 None] Delete backup finished, backup cbe785d6-2f9f-4944-9699-f1fe9c4db8ca deleted. Feb 05 21:09:15.626708 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Differential backup transfer completed in 8.0003s {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Feb 05 21:09:15.627406 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:09:15.655612 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:09:15.656417 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:09:15.680662 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:09:15.681964 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:09:15.703527 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Feb 05 21:09:15.706740 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Backing up metadata for volume 10f43b5e-5fc3-4f9e-be50-14c09c91d474. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:09:15.864306 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Backup '39685a0c-8fc6-4d89-ac0b-44ed5da25dab' of volume 10f43b5e-5fc3-4f9e-be50-14c09c91d474 finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:09:15.865887 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '10f43b5e-5fc3-4f9e-be50-14c09c91d474', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:15.909870 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] <== disconnect_volume: return (43ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:16.043383 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-137aa728-e349-49db-a0be-08a2fb1a2ba6 tempest-VolumesBackupsTest-350919582 None] Create backup finished. backup: 39685a0c-8fc6-4d89-ac0b-44ed5da25dab. Feb 05 21:09:18.482079 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Restore backup started, backup: 39685a0c-8fc6-4d89-ac0b-44ed5da25dab volume: 41f660e3-74ff-446e-84b9-f0721676e61f. Feb 05 21:09:18.528392 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:18.539067 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.560855 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.031s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:18.561970 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[97ba9131-a93f-49ba-9c15-acff611d8d85]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:18.563638 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.586805 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[db3849bc-0dea-409a-9289-ffdf7182c4b3]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:18.587321 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:18.588930 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[db3849bc-0dea-409a-9289-ffdf7182c4b3]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:18.591025 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.612708 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2bd34d4c-ad64-4403-925e-0dfc62920211]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:18.613456 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:18.614677 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bd34d4c-ad64-4403-925e-0dfc62920211]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:18.615568 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.642043 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.026s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:18.642613 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[d08ab4b5-f258-4a45-a01f-4c77874c54fb]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:18.644196 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.670191 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.025s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:18.671714 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[33f939bf-776d-4d0c-a644-b9a2223b9f27]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:18.676406 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.696347 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.022s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:18.697014 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[10b96040-c7ea-43d6-b5c0-720ea8333077]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:18.698109 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:18.738663 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] CMD "nvme version" returned: 0 in 0.040s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:18.740746 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] <== get_connector_properties: return (211ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:19.599586 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:09:19.601250 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-41f660e3-74ff-446e-84b9-f0721676e61f', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '41f660e3-74ff-446e-84b9-f0721676e61f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:19.602323 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:09:19.668392 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] <== connect_volume: return (65ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:19.680078 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:09:19.692968 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Starting restore from Ceph backup=39685a0c-8fc6-4d89-ac0b-44ed5da25dab to volume=volume-41f660e3-74ff-446e-84b9-f0721676e61f {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Feb 05 21:09:19.778881 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Looking for snapshot of backup base 'volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:09:19.779423 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Found snapshot 'backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:09:19.789828 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Volume file is RBD. {{(pid=122595) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Feb 05 21:09:19.845596 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Attempting differential restore. {{(pid=122595) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Feb 05 21:09:19.846250 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Attempting incremental restore from base='volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' snap='backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711' {{(pid=122595) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Feb 05 21:09:19.846755 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Performing differential transfer from 'volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' to 'volume-41f660e3-74ff-446e-84b9-f0721676e61f' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:09:19.847219 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab@backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:09:19.847671 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_je6e3u0j --pool volumes - volumes/volume-41f660e3-74ff-446e-84b9-f0721676e61f' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:09:25.680858 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Restore transfer completed in 5.8322s {{(pid=122595) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Feb 05 21:09:25.682122 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Feb 05 21:09:25.750822 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Restoring volume base metadata {{(pid=122595) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Feb 05 21:09:25.790788 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] No metadata of type 'volume-metadata' to restore {{(pid=122595) put /opt/stack/cinder/cinder/backup/driver.py:345}} Feb 05 21:09:25.791263 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Restoring volume glance metadata {{(pid=122595) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Feb 05 21:09:26.009205 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Restore to volume 41f660e3-74ff-446e-84b9-f0721676e61f finished successfully. {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Feb 05 21:09:26.011034 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-41f660e3-74ff-446e-84b9-f0721676e61f', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '41f660e3-74ff-446e-84b9-f0721676e61f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': False}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:26.023845 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] <== disconnect_volume: return (11ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:26.071594 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.manager [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Encryption key ID for volume 41f660e3-74ff-446e-84b9-f0721676e61f already matches encryption key ID in backup 39685a0c-8fc6-4d89-ac0b-44ed5da25dab. {{(pid=122595) _run_restore /opt/stack/cinder/cinder/backup/manager.py:824}} Feb 05 21:09:26.101952 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-f3c860bd-3b56-4dea-a68d-49226f713613 tempest-VolumesBackupsTest-350919582 None] Finished restoring backup 39685a0c-8fc6-4d89-ac0b-44ed5da25dab to volume 41f660e3-74ff-446e-84b9-f0721676e61f. Feb 05 21:09:30.372188 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Delete backup started, backup: 39685a0c-8fc6-4d89-ac0b-44ed5da25dab. Feb 05 21:09:30.374113 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Delete started for backup=39685a0c-8fc6-4d89-ac0b-44ed5da25dab {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:09:30.374834 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Trying diff format basename='volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' for backup base image of volume 10f43b5e-5fc3-4f9e-be50-14c09c91d474. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:09:30.440843 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Looking for snapshot of backup base 'volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:09:30.441670 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Found snapshot 'backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:09:30.442280 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Deleting backup snapshot='backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab.snap.1644095346.8835711' {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Feb 05 21:09:30.637936 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Deleting backup base image='volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' of volume 10f43b5e-5fc3-4f9e-be50-14c09c91d474. Feb 05 21:09:31.129932 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Base backup image='volume-10f43b5e-5fc3-4f9e-be50-14c09c91d474.backup.39685a0c-8fc6-4d89-ac0b-44ed5da25dab' of volume 10f43b5e-5fc3-4f9e-be50-14c09c91d474 deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:09:31.207863 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Delete of backup '39685a0c-8fc6-4d89-ac0b-44ed5da25dab' for volume '10f43b5e-5fc3-4f9e-be50-14c09c91d474' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:09:31.215365 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:09:31.244890 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Created reservations ['24ebf168-a273-48b4-bd5f-75bc3a38d011', '69eb0af3-3906-408f-b1c0-087db0b9373b'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:09:31.290734 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-2e90de4e-1029-4eef-a527-d879cd05711f tempest-VolumesBackupsTest-350919582 None] Delete backup finished, backup 39685a0c-8fc6-4d89-ac0b-44ed5da25dab deleted. Feb 05 21:09:31.548380 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:09:31.549453 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:09:43.754202 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Create backup started, backup: 9f4e66ef-8f16-49c7-9500-076004f0014d volume: 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea snapshot: 37619e5a-7495-43b1-bc07-3cb25b99c0bd. Feb 05 21:09:43.766427 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:09:43Z,data_timestamp=2022-02-05T21:09:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-1074229487',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=9f4e66ef-8f16-49c7-9500-076004f0014d,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='c04615402925462792ca03f711db23c1',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='37619e5a-7495-43b1-bc07-3cb25b99c0bd',status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='6682477b172d460881a96d68f6bd053c',volume_id=419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea) Feb 05 21:09:44.468447 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:44.471973 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.484037 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:44.484461 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[22150274-522d-4fb5-9838-9fe20a3cf249]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:44.486290 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.499366 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c8bc98b4-e143-40b4-9b17-58b30ae33093]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:44.499896 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:44.500956 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8bc98b4-e143-40b4-9b17-58b30ae33093]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:44.502885 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.519920 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[08a94840-5747-4432-b4cb-a0a438c52830]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:09:44.520222 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:09:44.521439 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[08a94840-5747-4432-b4cb-a0a438c52830]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:44.526697 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.546294 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:44.546732 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[54420731-7c09-4a0d-badd-ccd69896e362]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:44.550342 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.564843 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.015s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:44.566136 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b957907-fdc8-453f-8846-3a36b989d398]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:44.572459 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.609410 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.041s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:44.609706 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[182c959e-1b46-41be-834b-f52be0227953]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:09:44.610972 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:09:44.655464 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] CMD "nvme version" returned: 0 in 0.044s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:09:44.657451 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] <== get_connector_properties: return (188ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:44.719912 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-c02eea2c-8f6b-49d2-a835-a0183021929b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:09:47.462334 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:09:47.468455 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-baa93d66-524e-4edc-addd-c5c2d6b460a4', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'baa93d66-524e-4edc-addd-c5c2d6b460a4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:09:47.471322 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:09:47.604796 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] <== connect_volume: return (135ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:09:47.644171 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Starting backup of volume='419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:09:47.718841 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:09:47.719322 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Creating backup base image='volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.base' for volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea. {{(pid=122595) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}} Feb 05 21:09:47.812479 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Copying data from volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea. {{(pid=122595) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}} Feb 05 21:09:47.848430 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferring data between 'volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea' and 'volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.base' {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}} Feb 05 21:09:47.848851 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] 8 chunks of 134217728 bytes to be transferred {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}} Feb 05 21:09:49.283630 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 1 of 8 (91400K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:09:50.634148 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 2 of 8 (97156K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:09:52.147794 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 3 of 8 (86720K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:09:53.664677 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 4 of 8 (86438K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:09:55.263644 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 5 of 8 (82352K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:09:56.990191 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 6 of 8 (75981K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:09:58.719620 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 7 of 8 (75814K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:10:00.433620 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Transferred chunk 8 of 8 (76499K/s) {{(pid=122595) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Feb 05 21:10:00.503886 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:10:00.522953 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:10:00.523372 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:10:00.533200 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:10:00.534581 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:10:00.565046 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] No metadata type 'volume-glance-metadata' available {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Feb 05 21:10:00.565619 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Backing up metadata for volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:10:00.646233 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Backup '9f4e66ef-8f16-49c7-9500-076004f0014d' of volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:10:00.647811 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-baa93d66-524e-4edc-addd-c5c2d6b460a4', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'baa93d66-524e-4edc-addd-c5c2d6b460a4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:10:00.670527 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] <== disconnect_volume: return (22ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:10:00.835948 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-f3029c8c-7abd-4edc-aa8b-2d3925d8a748 tempest-VolumesSnapshotTestJSON-1761633007 None] Create backup finished. backup: 9f4e66ef-8f16-49c7-9500-076004f0014d. Feb 05 21:10:01.577567 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Delete backup started, backup: 9f4e66ef-8f16-49c7-9500-076004f0014d. Feb 05 21:10:01.581621 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Delete started for backup=9f4e66ef-8f16-49c7-9500-076004f0014d {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:10:01.586621 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Trying diff format basename='volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.9f4e66ef-8f16-49c7-9500-076004f0014d' for backup base image of volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:10:01.673719 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Image 'volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.9f4e66ef-8f16-49c7-9500-076004f0014d' not found - trying diff format name {{(pid=122595) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}} Feb 05 21:10:01.699537 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Looking for snapshot of backup base 'volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.base' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:10:01.699977 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Backup base 'volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.base' has no snapshots {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}} Feb 05 21:10:01.700355 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] No backup snapshot to delete {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Feb 05 21:10:01.708955 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Deleting backup base image='volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.base' of volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea. Feb 05 21:10:02.555037 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Base backup image='volume-419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea.backup.base' of volume 419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:10:02.643240 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Delete of backup '9f4e66ef-8f16-49c7-9500-076004f0014d' for volume '419ad4d0-6b45-4fb7-9e6f-920b37e4a0ea' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:10:02.654652 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:10:02.720776 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Created reservations ['0b1fad1e-b785-4356-ab48-737c4bd77652', '38d83992-8e86-4d61-9e9e-b9de111c3cbd'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:10:02.759092 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-132a6d93-4ffb-4802-b9ef-bcbfed257032 tempest-VolumesSnapshotTestJSON-1761633007 None] Delete backup finished, backup 9f4e66ef-8f16-49c7-9500-076004f0014d deleted. Feb 05 21:10:31.548734 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:10:31.549544 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:11:31.544441 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:11:31.545112 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:12:31.549863 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:12:31.550629 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:13:31.554848 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:13:31.555567 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:14:31.552663 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:14:31.553392 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:15:31.549443 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:15:31.550056 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:16:31.551134 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:16:31.551862 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:17:31.548356 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:17:31.549100 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:18:31.552971 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:18:31.553643 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:19:31.557673 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:19:31.558288 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:20:31.562506 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:20:31.563213 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:21:31.558838 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:21:31.559527 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:22:31.561156 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:22:31.561853 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:23:31.566575 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:23:31.567257 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:24:31.563373 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:24:31.564090 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:25:31.560269 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:25:31.560956 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:26:31.566276 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:26:31.566986 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:27:31.562809 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:27:31.563442 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:28:31.567400 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:28:31.568086 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:29:31.572274 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:29:31.572959 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:30:31.577717 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:30:31.578779 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:31:31.575471 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:31:31.576192 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:32:08.415530 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Create backup started, backup: a0679c23-44a4-4396-9299-89991fb4b729 volume: 894fa68e-2bf7-4768-ac8b-021977d33c46. Feb 05 21:32:08.422758 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-05T21:32:08Z,data_timestamp=2022-02-05T21:32:08Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=a0679c23-44a4-4396-9299-89991fb4b729,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='457bdfbbba5b4e69a1bf1577ce565fc9',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='dd304cad36bd4b3188051c354d575206',volume_id=894fa68e-2bf7-4768-ac8b-021977d33c46) Feb 05 21:32:08.463775 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:32:08.465977 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.486766 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.021s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:08.487292 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c5d975a-105d-448b-96e0-57be09024916]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:08.488985 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.496688 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b7f912c7-c034-4a5e-a875-45ad0b2cee29]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:32:08.497707 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:32:08.499398 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7f912c7-c034-4a5e-a875-45ad0b2cee29]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:08.503426 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.510580 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3a029cbb-822c-40b9-8108-0682b05ad6ed]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:32:08.511601 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a029cbb-822c-40b9-8108-0682b05ad6ed]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:08.515558 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.550478 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.035s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:08.550839 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cdb83fb-1a74-4599-aba5-9037f3b5164d]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:08.552003 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.561132 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:08.561597 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[03d65910-1df1-489e-a055-d8767f7296c4]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:08.562801 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.570143 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:08.570587 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed1e2ef3-b4f2-47dc-9576-719763035f82]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:08.571802 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:08.592294 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:08.593415 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:32:09.305512 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:32:09.307216 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-894fa68e-2bf7-4768-ac8b-021977d33c46', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '894fa68e-2bf7-4768-ac8b-021977d33c46', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:32:09.308273 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:32:09.375027 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] <== connect_volume: return (66ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:32:09.383608 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:32:09.400603 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Starting backup of volume='894fa68e-2bf7-4768-ac8b-021977d33c46'. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Feb 05 21:32:09.401630 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Volume file is RBD: attempting optimized backup {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Feb 05 21:32:09.423700 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Creating base image 'volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' {{(pid=122595) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Feb 05 21:32:09.424233 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=122595) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Feb 05 21:32:09.456519 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Using --from-snap 'None' for incremental backup of volume 894fa68e-2bf7-4768-ac8b-021977d33c46. {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Feb 05 21:32:09.456933 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Creating backup snapshot='backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458' {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Feb 05 21:32:09.887159 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Performing differential transfer from 'volume-894fa68e-2bf7-4768-ac8b-021977d33c46' to 'volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:32:09.887511 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_rbsna5ax --pool volumes volumes/volume-894fa68e-2bf7-4768-ac8b-021977d33c46@backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:32:09.887892 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:32:14.615211 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Differential backup transfer completed in 4.7278s {{(pid=122595) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Feb 05 21:32:14.615940 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Getting metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Feb 05 21:32:14.628792 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=122595) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Feb 05 21:32:14.629211 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Getting metadata type 'volume-metadata' {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Feb 05 21:32:14.635333 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] No metadata type 'volume-metadata' available {{(pid=122595) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Feb 05 21:32:14.635723 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Getting metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Feb 05 21:32:14.642600 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=122595) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Feb 05 21:32:14.643681 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Backing up metadata for volume 894fa68e-2bf7-4768-ac8b-021977d33c46. {{(pid=122595) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Feb 05 21:32:14.673816 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Backup 'a0679c23-44a4-4396-9299-89991fb4b729' of volume 894fa68e-2bf7-4768-ac8b-021977d33c46 finished. {{(pid=122595) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Feb 05 21:32:14.675567 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-894fa68e-2bf7-4768-ac8b-021977d33c46', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '894fa68e-2bf7-4768-ac8b-021977d33c46', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': True}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:32:14.695834 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] <== disconnect_volume: return (19ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:32:14.752504 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-2b943691-d019-41f8-a802-e19a756aed02 tempest-TestVolumeBackupRestore-1373251088 None] Create backup finished. backup: a0679c23-44a4-4396-9299-89991fb4b729. Feb 05 21:32:16.888411 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Restore backup started, backup: a0679c23-44a4-4396-9299-89991fb4b729 volume: 23710495-8762-4ece-a46a-b17509d91aa5. Feb 05 21:32:16.902048 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:32:16.903510 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.913839 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.010s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:16.914573 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[23e3b9be-4730-41e8-9bf0-4ea211535a9c]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ee59caef4a72\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:16.916037 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.921893 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[df5755c5-ddf8-4f3c-937d-c1e2f417c8b2]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:32:16.922467 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:32:16.923761 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[df5755c5-ddf8-4f3c-937d-c1e2f417c8b2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:16.924096 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.929492 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[fe65dcd2-75ce-4906-82cd-88f8a03e5ff7]: [Errno 2] No such file or directory: 'systool' {{(pid=144658) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: Traceback (most recent call last): Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: ret = func(*f_args, **f_kwargs) Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: return func(*args, **kwargs) Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: return putils.execute(on_execute=on_execute, Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: obj = subprocess.Popen(cmd, Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: self._execute_child(args, executable, preexec_fn, close_fds, Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: raise child_exception_type(errno_num, err_msg, err_filename) Feb 05 21:32:16.930197 ncdv-master-1 cinder-backup[122595]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Feb 05 21:32:16.931486 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe65dcd2-75ce-4906-82cd-88f8a03e5ff7]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:16.932224 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.940702 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:16.941187 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e177169-a11b-44d3-a282-8cf0e65b42ee]: (4, ('/dev/sda1\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:16.942350 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.950847 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:16.951188 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[c70f4a06-0f3c-4c5f-b587-0ad27b00e68e]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:16.952372 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.959651 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=144658) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:16.959986 ncdv-master-1 cinder-backup[122595]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a600cb1-da67-4c0a-8450-5c7dcd62dfc5]: (4, ('cf4c6349-9cfc-5a4e-820b-074aead70b7d\n', '')) {{(pid=144658) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Feb 05 21:32:16.960781 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Running cmd (subprocess): nvme version {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Feb 05 21:32:16.980829 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_concurrency.processutils [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=122595) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Feb 05 21:32:16.981899 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.utils [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] <== get_connector_properties: return (78ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.32', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ee59caef4a72', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'cf4c6349-9cfc-5a4e-820b-074aead70b7d', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:56693fff-cd20-463d-a23a-f7be0fe3887f'} {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:32:17.647933 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connector [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Factory for rbd on None {{(pid=122595) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Feb 05 21:32:17.649470 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-23710495-8762-4ece-a46a-b17509d91aa5', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '23710495-8762-4ece-a46a-b17509d91aa5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:32:17.650551 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.linuxrbd [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] opening connection to ceph cluster (timeout=-1). {{(pid=122595) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Feb 05 21:32:17.689293 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] <== connect_volume: return (39ms) {'path': } {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:32:17.697415 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:32:17.708317 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Starting restore from Ceph backup=a0679c23-44a4-4396-9299-89991fb4b729 to volume=volume-23710495-8762-4ece-a46a-b17509d91aa5 {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Feb 05 21:32:17.764486 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Looking for snapshot of backup base 'volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:32:17.764889 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Found snapshot 'backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:32:17.769750 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Volume file is RBD. {{(pid=122595) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Feb 05 21:32:17.800427 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Attempting differential restore. {{(pid=122595) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Feb 05 21:32:17.801008 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Attempting incremental restore from base='volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' snap='backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458' {{(pid=122595) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Feb 05 21:32:17.801423 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Performing differential transfer from 'volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' to 'volume-23710495-8762-4ece-a46a-b17509d91aa5' {{(pid=122595) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Feb 05 21:32:17.801867 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729@backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458 -' into... {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Feb 05 21:32:17.802181 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_3qv8og45 --pool volumes - volumes/volume-23710495-8762-4ece-a46a-b17509d91aa5' {{(pid=122595) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Feb 05 21:32:20.978696 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Restore transfer completed in 3.1767s {{(pid=122595) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Feb 05 21:32:20.979723 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Feb 05 21:32:21.001810 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Restoring volume base metadata {{(pid=122595) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Feb 05 21:32:21.022716 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] No metadata of type 'volume-metadata' to restore {{(pid=122595) put /opt/stack/cinder/cinder/backup/driver.py:345}} Feb 05 21:32:21.023107 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.driver [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Restoring volume glance metadata {{(pid=122595) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Feb 05 21:32:21.153320 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Restore to volume 23710495-8762-4ece-a46a-b17509d91aa5 finished successfully. {{(pid=122595) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Feb 05 21:32:21.154993 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-23710495-8762-4ece-a46a-b17509d91aa5', 'hosts': ['10.222.0.32'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '23710495-8762-4ece-a46a-b17509d91aa5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, ': False}" {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Feb 05 21:32:21.164456 ncdv-master-1 cinder-backup[122595]: DEBUG os_brick.initiator.connectors.rbd [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] <== disconnect_volume: return (9ms) None {{(pid=122595) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Feb 05 21:32:21.212409 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.manager [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Encryption key ID for volume 23710495-8762-4ece-a46a-b17509d91aa5 already matches encryption key ID in backup a0679c23-44a4-4396-9299-89991fb4b729. {{(pid=122595) _run_restore /opt/stack/cinder/cinder/backup/manager.py:824}} Feb 05 21:32:21.241025 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-dacab71f-cd86-46da-af62-410a104acd64 tempest-TestVolumeBackupRestore-1373251088 None] Finished restoring backup a0679c23-44a4-4396-9299-89991fb4b729 to volume 23710495-8762-4ece-a46a-b17509d91aa5. Feb 05 21:32:31.580438 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:32:31.581259 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:33:26.058158 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Delete backup started, backup: a0679c23-44a4-4396-9299-89991fb4b729. Feb 05 21:33:26.061526 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Delete started for backup=a0679c23-44a4-4396-9299-89991fb4b729 {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Feb 05 21:33:26.061963 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Trying diff format basename='volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' for backup base image of volume 894fa68e-2bf7-4768-ac8b-021977d33c46. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Feb 05 21:33:26.118576 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Looking for snapshot of backup base 'volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Feb 05 21:33:26.119542 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Found snapshot 'backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458' {{(pid=122595) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Feb 05 21:33:26.120269 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Deleting backup snapshot='backup.a0679c23-44a4-4396-9299-89991fb4b729.snap.1644096729.4565458' {{(pid=122595) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Feb 05 21:33:26.922793 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Deleting backup base image='volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' of volume 894fa68e-2bf7-4768-ac8b-021977d33c46. Feb 05 21:33:27.444424 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Base backup image='volume-894fa68e-2bf7-4768-ac8b-021977d33c46.backup.a0679c23-44a4-4396-9299-89991fb4b729' of volume 894fa68e-2bf7-4768-ac8b-021977d33c46 deleted. {{(pid=122595) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Feb 05 21:33:27.480555 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.backup.drivers.ceph [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Delete of backup 'a0679c23-44a4-4396-9299-89991fb4b729' for volume '894fa68e-2bf7-4768-ac8b-021977d33c46' finished. {{(pid=122595) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Feb 05 21:33:27.486787 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_db.sqlalchemy.engines [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122595) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Feb 05 21:33:27.510290 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.quota [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Created reservations ['684c8ae7-4b50-4eae-8456-eb52f8c75ec7', '6c37b9d4-e1d4-40b9-b107-798c1c680415'] {{(pid=122595) reserve /opt/stack/cinder/cinder/quota.py:755}} Feb 05 21:33:27.535576 ncdv-master-1 cinder-backup[122595]: INFO cinder.backup.manager [None req-7327f06c-d6dd-4d61-a68c-5d9fc3534cb5 tempest-TestVolumeBackupRestore-1373251088 None] Delete backup finished, backup a0679c23-44a4-4396-9299-89991fb4b729 deleted. Feb 05 21:33:31.577207 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:33:31.577736 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:34:31.583023 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:34:31.583733 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:35:31.579020 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:35:31.579642 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:36:31.584338 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:36:31.585283 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:37:31.585207 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:37:31.585933 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:38:31.583568 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:38:31.584265 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:40:31.589674 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:40:31.590310 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:41:31.595109 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:41:31.595838 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:42:31.591368 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:42:31.592110 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:43:31.595873 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:43:31.596601 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:44:31.601468 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:44:31.602165 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:45:31.601984 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:45:31.602764 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:46:31.599090 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:46:31.599715 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:47:31.604185 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:47:31.604907 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:48:31.600596 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:48:31.601296 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:49:31.605794 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:49:31.606509 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:50:31.610789 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:50:31.611407 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:51:31.608342 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:51:31.609119 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:52:31.613260 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:52:31.613965 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:53:31.609873 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:53:31.610601 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:54:31.615737 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:54:31.616460 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:55:31.621798 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:55:31.622898 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:56:31.627138 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:56:31.627861 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:57:31.632400 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:57:31.633369 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:58:31.629586 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:58:31.630301 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 21:59:31.626813 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 21:59:31.627510 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:00:31.631748 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:00:31.632420 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:01:31.637641 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:01:31.638317 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:02:31.642984 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:02:31.643643 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:03:31.647957 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:03:31.648600 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:04:31.653039 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:04:31.653740 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:05:31.658039 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:05:31.658809 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:06:31.659565 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:06:31.660271 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:07:31.665069 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:07:31.665796 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:08:31.670321 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:08:31.671046 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:09:31.676004 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:09:31.676748 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:10:31.680646 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:10:31.681386 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:11:31.685490 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:11:31.686261 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:12:31.681358 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:12:31.682070 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:13:31.678051 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:13:31.678796 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:14:31.683293 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:14:31.684029 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:15:31.688915 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:15:31.689633 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:16:31.685689 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:16:31.686450 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:17:31.681537 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:17:31.682262 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Feb 05 22:18:31.686662 ncdv-master-1 cinder-backup[122595]: DEBUG oslo_service.periodic_task [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=122595) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Feb 05 22:18:31.687390 ncdv-master-1 cinder-backup[122595]: DEBUG cinder.manager [None req-9ff91575-6d63-4560-a89f-8eddbf45f173 None None] Notifying Schedulers of capabilities ... {{(pid=122595) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}