-- Logs begin at Sat 2022-02-12 20:01:13 UTC, end at Sat 2022-02-12 22:21:56 UTC. --
Feb 12 20:52:38.838265 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service.
Feb 12 20:52:42.282225 ncdv-master-1 cinder-backup[134160]: INFO cinder.cmd.backup [-] Backup running with 1 processes.
Feb 12 20:52:42.283737 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=134160) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}}
Feb 12 20:52:42.284028 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=134160) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}}
Feb 12 20:52:42.444683 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_db.sqlalchemy.engines [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 20:52:42.678934 ncdv-master-1 cinder-backup[134160]: INFO cinder.rpc [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version.
Feb 12 20:52:42.682726 ncdv-master-1 cinder-backup[134160]: INFO cinder.rpc [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Automatically selected cinder-backup RPC version 2.3 as minimum service version.
Feb 12 20:52:42.686572 ncdv-master-1 cinder-backup[134160]: INFO cinder.rpc [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version.
Feb 12 20:52:42.691161 ncdv-master-1 cinder-backup[134160]: INFO cinder.rpc [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Automatically selected cinder-volume RPC version 3.17 as minimum service version.
Feb 12 20:52:42.699147 ncdv-master-1 cinder-backup[134160]: INFO cinder.rpc [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
Feb 12 20:52:42.703679 ncdv-master-1 cinder-backup[134160]: INFO cinder.rpc [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version.
Feb 12 20:52:42.767334 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_db.sqlalchemy.engines [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 20:52:42.865286 ncdv-master-1 cinder-backup[134160]: INFO oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Starting 1 workers
Feb 12 20:52:42.872551 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Started child 134603 {{(pid=134160) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Feb 12 20:52:42.879278 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Full set of CONF: {{(pid=134160) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}}
Feb 12 20:52:42.879746 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] ******************************************************************************** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}}
Feb 12 20:52:42.880067 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] Configuration options gathered from: {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}}
Feb 12 20:52:42.881718 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}}
Feb 12 20:52:42.882410 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}}
Feb 12 20:52:42.882703 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] ================================================================================ {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}}
Feb 12 20:52:42.884698 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] allow_availability_zone_fallback = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.885097 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] allow_compression_on_image_upload = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.885631 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] allowed_direct_url_schemes     = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.888904 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] api_rate_limit                 = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.890549 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] auth_strategy                  = keystone {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.891043 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_availability_zone      = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.891441 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_api_class               = cinder.backup.api.API {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.892345 ncdv-master-1 cinder-backup[134603]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0)
Feb 12 20:52:42.893602 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_chunk_size         = 134217728 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.895951 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_conf               = /etc/ceph/ceph.conf {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.898511 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_image_journals     = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.898511 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_pool               = backups {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.898511 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_stripe_count       = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.898511 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_stripe_unit        = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.899111 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_ceph_user               = cinder-bak {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.899492 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_driver                  = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.902296 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_driver_init_check_interval = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.902710 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_driver_stats_polling_interval = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.903083 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_manager                 = cinder.backup.manager.BackupManager {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.904296 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_max_operations          = 15 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.904750 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_metadata_version        = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.905155 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_name_template           = backup-%s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.905540 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_native_threads_pool_size = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.905943 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_object_number_per_notification = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.906296 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_service_inithost_offload = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.906667 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_timer_interval          = 120 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.907051 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_use_same_host           = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.907950 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_use_temp_snapshot       = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.914465 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backup_workers                 = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.916104 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] chap_password                  = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.916545 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] chap_username                  =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.916946 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.918551 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_internal_tenant_project_id = 1b89bfd9b7634100a4d518cb7b8b66f2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.918937 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_internal_tenant_user_id = 856df892e65c41e9b2ac29de966edec5 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.920458 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] compression_format             = gzip {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.920891 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] config_dir                     = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.922417 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] config_file                    = ['/etc/cinder/cinder.conf'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.922876 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] config_source                  = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.928723 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] consistencygroup_api_class     = cinder.consistencygroup.api.API {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.929155 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] control_exchange               = openstack {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.930069 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] debug                          = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.930439 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] default_availability_zone      = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.930796 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] default_group_type             = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.931415 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.933523 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] default_volume_type            = ceph {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.933914 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] driver_client_cert             = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.934276 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] driver_client_cert_key         = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.934648 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] driver_data_namespace          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.936185 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] driver_ssl_cert_path           = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.936706 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] driver_ssl_cert_verify         = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.937052 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] driver_use_ssl                 = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.937409 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] enable_new_services            = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.937768 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] enable_unsupported_driver      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.938156 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] enabled_backends               = ['ceph'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.941558 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] enforce_multipath_for_image_xfer = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.942013 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] filter_function                = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.942386 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_api_insecure            = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.945855 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_api_servers             = ['http://10.222.0.31/image'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.946244 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_api_ssl_compression     = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.946611 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_ca_certificates_file    = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.947016 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_catalog_info            = image:glance:publicURL {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.947386 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_certfile                = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.947913 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.949523 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_keyfile                 = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.950386 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_num_retries             = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.950908 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] glance_request_timeout         = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.951270 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] goodness_function              = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.951656 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] graceful_shutdown_timeout      = 5 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.952024 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] group_api_class                = cinder.group.api.API {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.952446 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] host                           = ncdv-master-1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.952967 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iet_conf                       = /etc/iet/ietd.conf {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.953394 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_compress_on_upload       = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.953748 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_conversion_address_space_limit = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.954119 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_conversion_cpu_limit     = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.954866 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_conversion_dir           = /opt/stack/data/cinder/conversion {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.963514 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_upload_use_cinder_backend = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.963898 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_upload_use_internal_tenant = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.964264 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_volume_cache_enabled     = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.964734 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_volume_cache_max_count   = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.965111 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] image_volume_cache_max_size_gb = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.966559 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.lockutils [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] Lock "cleanup_incomplete_backups_134160" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.004s {{(pid=134603) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}}
Feb 12 20:52:42.966893 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] Cleaning up incomplete backup operations.
Feb 12 20:52:42.968647 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] instance_format                = [instance: %(uuid)s]  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.969336 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.969781 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iscsi_iotype                   = fileio {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.971766 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iscsi_secondary_ip_addresses   = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.972145 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iscsi_target_flags             =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.973290 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iscsi_write_cache              = on {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.973690 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iser_helper                    = tgtadm {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.976854 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iser_ip_address                = 10.222.0.31 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.977295 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iser_port                      = 3260 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.977702 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] iser_target_prefix             = iqn.2010-10.org.openstack: {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.978097 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_config_append              = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.978460 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.978817 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_dir                        = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.981855 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_file                       = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.982177 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_options                    = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.982549 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_rotate_interval            = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.983164 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_rotate_interval_type       = days {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.989206 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] log_rotation_type              = none {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.990381 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 20:52:42.990881 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.991437 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.992199 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.992576 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.992873 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.993250 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] max_age                        = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.995936 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] max_logfile_count              = 30 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.996313 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] max_logfile_size_mb            = 200 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.996697 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] max_over_subscription_ratio    = 20.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.997080 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] message_reap_interval          = 86400 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.997466 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] message_ttl                    = 2592000 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.997819 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] monkey_patch                   = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.998178 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] monkey_patch_modules           = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:42.998470 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] my_ip                          = 10.222.0.31 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.003812 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] no_snapshot_gb_quota           = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.004234 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] num_iser_scan_tries            = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.004683 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] num_shell_tries                = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.007738 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] num_volume_device_scan_tries   = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.008117 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] nvmet_ns_id                    = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.008492 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] nvmet_port_id                  = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.009189 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_max_limit                = 1000 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.009816 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_volume_ext_list          = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.010240 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.010625 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_volume_listen            = 0.0.0.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.011054 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_volume_listen_port       = 8776 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.013734 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_volume_use_ssl           = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.014153 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] osapi_volume_workers           = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.014529 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] per_volume_size_limit          = -1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.014888 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] periodic_fuzzy_delay           = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.015264 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] periodic_interval              = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.015631 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] public_endpoint                = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.019277 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] publish_errors                 = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.020045 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_backup_gigabytes         = 1000 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.020432 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_backups                  = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.021191 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_consistencygroups        = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.022739 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_driver                   = cinder.quota.DbQuotaDriver {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.023112 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_gigabytes                = 1000 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.023473 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_groups                   = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.030773 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_snapshots                = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.031196 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] quota_volumes                  = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.031569 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] rate_limit_burst               = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.032144 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] rate_limit_except_level        = CRITICAL {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.033414 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] rate_limit_interval            = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.034010 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] replication_device             = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.034494 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] report_discard_supported       = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.034860 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] report_interval                = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.035307 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] reservation_clean_interval     = 86400 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.035609 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] reservation_expire             = 86400 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.035969 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] reserved_percentage            = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.036359 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] resource_query_filters_file    = /etc/cinder/resource_filters.json {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.039992 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] restore_discard_excess_bytes   = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.040958 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] rootwrap_config                = /etc/cinder/rootwrap.conf {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.041393 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] rpc_response_timeout           = 120 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.041952 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] run_external_periodic_tasks    = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.042349 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] scheduler_manager              = cinder.scheduler.manager.SchedulerManager {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.042720 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] scst_target_driver             = iscsi {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.043067 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] scst_target_iqn_name           = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.044350 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_down_time              = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.044761 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] snapshot_name_template         = snapshot-%s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.045116 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] split_loggers                  = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.045433 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] state_path                     = /opt/stack/data/cinder {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.045801 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] storage_availability_zone      = nova {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.046169 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] storage_protocol               = iscsi {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.046551 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] syslog_log_facility            = LOG_USER {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.046907 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] target_helper                  = lioadm {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.047377 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] target_ip_address              = 10.222.0.31 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.047749 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] target_port                    = 3260 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.048200 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] target_prefix                  = iqn.2010-10.org.openstack: {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.049706 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] target_protocol                = iscsi {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.050088 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] trace_flags                    = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.050450 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] transfer_api_class             = cinder.transfer.api.API {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.061660 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] transport_url                  = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.061994 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] until_refresh                  = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.062585 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_chap_auth                  = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.062950 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_default_quota_class        = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.063336 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_eventlog                   = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.064478 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_journal                    = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.064944 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_json                       = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.070116 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_multipath_for_image_xfer   = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.070700 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_stderr                     = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.072516 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] use_syslog                     = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.072934 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] verify_glance_signatures       = enabled {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.073333 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_api_class               = cinder.volume.api.API {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.073912 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_backend_name            = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.074301 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_clear                   = zero {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.074810 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_clear_ionice            = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.075130 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_clear_size              = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.075501 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_copy_blkio_cgroup_name  = cinder-volume-copy {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.075857 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_copy_bps_limit          = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.076219 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_dd_blocksize            = 1M {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.076582 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_manager                 = cinder.volume.manager.VolumeManager {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.076942 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_name_template           = volume-%s {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.077305 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volume_usage_audit_period      = month {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.077773 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] volumes_dir                    = /opt/stack/data/cinder/volumes {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.078115 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] watch_log_file                 = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Feb 12 20:52:43.078614 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.079028 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.079494 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.enforce_new_defaults = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.080689 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.enforce_scope      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.081080 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.policy_default_rule = default {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.081620 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.082032 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.policy_file        = policy.yaml {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.082931 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.083307 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.083672 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.084217 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.084847 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.085236 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_concurrency.disable_process_locking = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.085628 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_concurrency.lock_path     = /opt/stack/data/cinder {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.086070 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.admin_password = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.086434 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.086781 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.admin_token = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.087127 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.admin_user  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.087523 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_admin_prefix =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.087882 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_host   = 127.0.0.1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.088246 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_port   = 35357 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.088621 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_protocol = https {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.089007 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_section = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.089411 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_type   = password {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.089779 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_uri    = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.090138 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.auth_version = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.090590 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.cache       = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.091130 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.cafile      = /opt/stack/data/ca-bundle.pem {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.091475 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.certfile    = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.091819 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.delay_auth_decision = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.092204 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.093106 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.http_connect_timeout = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.093483 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.094002 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.identity_uri = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.094470 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.include_service_catalog = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.095423 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.insecure    = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.096406 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.interface   = public {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.096736 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.keyfile     = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.097787 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.098144 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.098494 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.098847 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.099195 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.099542 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.099923 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.100267 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.100667 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.101396 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.region_name = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.101768 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.102124 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.service_token_roles_required = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.103485 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.service_type = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.103861 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.token_cache_time = 300 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.104410 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.105016 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.backend                  = dogpile.cache.null {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.105429 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.backend_argument         = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.105801 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.config_prefix            = cache.oslo {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.106204 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.dead_timeout             = 60.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.106577 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.debug_cache_backend      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.106934 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.enable_retry_client      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.107283 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.enable_socket_keepalive  = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.107632 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.enabled                  = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.108006 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.expiration_time          = 600 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.108364 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.hashclient_retry_attempts = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.108721 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.hashclient_retry_delay   = 1.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.109074 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_dead_retry      = 300 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.109464 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.109814 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.110183 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_pool_maxsize    = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.110577 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.112019 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.112425 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.113207 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.proxies                  = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.113821 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.retry_attempts           = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.114447 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.retry_delay              = 0.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.115221 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.socket_keepalive_count   = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.115599 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.socket_keepalive_idle    = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.115951 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.socket_keepalive_interval = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.116296 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.tls_allowed_ciphers      = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.116671 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.tls_cafile               = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.117640 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.tls_certfile             = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.117640 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.tls_enabled              = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.117981 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cache.tls_keyfile              = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.118387 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] coordination.backend_url       = etcd3+http://10.222.0.31:2379 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.118828 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.backend               = sqlalchemy {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.119125 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.connection            = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.119508 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.connection_debug      = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.119873 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.connection_parameters =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.120222 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.connection_recycle_time = 3600 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.120571 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.connection_trace      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.120920 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.db_inc_retry_interval = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.121265 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.db_max_retries        = 20 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.121952 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.db_max_retry_interval = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.122300 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.db_retry_interval     = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.122670 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.max_overflow          = 50 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.123461 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.max_pool_size         = 5 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.124099 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.max_retries           = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.124705 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.mysql_enable_ndb      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.125084 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.125615 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.pool_timeout          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.125983 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.retry_interval        = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.126877 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.slave_connection      = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.127931 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.sqlite_synchronous    = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.128302 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] database.use_db_reconnect      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.128687 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.129043 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.129696 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.130062 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.130413 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.130821 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.connection_string     = messaging:// {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.131112 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.enabled               = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.131607 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.es_doc_type           = notification {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.132027 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.es_scroll_size        = 10000 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.132394 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.es_scroll_time        = 2m {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.132746 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.filter_error_trace    = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.133650 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.134271 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.sentinel_service_name = mymaster {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.134891 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.socket_timeout        = 0.1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.135244 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] profiler.trace_sqlalchemy      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.136422 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] privsep_osbrick.capabilities   = [21] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.136812 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] privsep_osbrick.group          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.137174 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] privsep_osbrick.helper_command = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.137576 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] privsep_osbrick.logger_name    = oslo_privsep.daemon {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.137930 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.138282 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] privsep_osbrick.user           = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.138676 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_sys_admin.capabilities  = [0, 1, 2, 3, 12, 21] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.139023 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_sys_admin.group         = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.139360 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_sys_admin.helper_command = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.139712 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_sys_admin.logger_name   = oslo_privsep.daemon {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.140055 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.140396 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] cinder_sys_admin.user          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.140788 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.auth_section      = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.141164 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.auth_type         = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.141532 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.cafile            = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.142255 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.certfile          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.143335 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.collect_timing    = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.144237 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.insecure          = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.144900 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.keyfile           = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.145277 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.send_service_user_token = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.145796 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.split_loggers     = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.146143 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] service_user.timeout           = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.146554 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.backend_availability_zone = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.146943 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.chap_password = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.147299 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.chap_username =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.147654 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.148000 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.148352 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.148690 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.driver_client_cert = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.149053 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.driver_client_cert_key = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.149424 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.driver_data_namespace = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.149766 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.150116 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.150455 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.driver_use_ssl = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.150817 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.enable_deferred_deletion = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.151162 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.enable_unsupported_driver = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.151511 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.152089 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.filter_function = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.152433 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.goodness_function = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.152788 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iet_conf      = /etc/iet/ietd.conf {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.153149 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.153628 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.154383 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.154920 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.155272 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.155719 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iscsi_iotype  = fileio {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.156101 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.156464 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iscsi_target_flags =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.156831 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iscsi_write_cache = on {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.157188 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iser_helper   = tgtadm {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.158833 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iser_ip_address = 10.222.0.31 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.159207 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iser_port     = 3260 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.159572 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.159962 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.160311 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.160658 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.num_shell_tries = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.161002 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.161482 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.nvmet_ns_id   = 10 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.161713 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.nvmet_port_id = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.162439 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.162805 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rados_connection_interval = 5 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.163159 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rados_connection_retries = 3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.164637 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_ceph_conf =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.165276 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.165702 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.166188 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.166543 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.166915 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_pool      = rbd {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.167256 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_secret_uuid = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.167606 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.167943 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.rbd_user      = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.168291 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.168629 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.replication_device = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.168972 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.report_discard_supported = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.169314 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.169988 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.reserved_percentage = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.170352 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.scst_target_driver = iscsi {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.170715 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.scst_target_iqn_name = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.171050 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.storage_protocol = iscsi {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.171407 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.target_helper = tgtadm {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.171875 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.target_ip_address = 10.222.0.31 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.172335 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.target_port   = 3260 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.172688 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.173073 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.target_protocol = iscsi {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.174574 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.trace_flags   = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.174950 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.unique_fqdn_network = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.175468 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.use_chap_auth = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.175820 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.176275 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_backend_name = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.176641 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_clear  = zero {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.176983 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_clear_ionice = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.177325 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_clear_size = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.177707 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.178049 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.178473 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.178929 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] backend_defaults.volumes_dir   = /opt/stack/data/cinder/volumes {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.179304 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_reports.file_event_handler = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.179671 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.180033 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_reports.log_dir           = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.180459 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.180827 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.181197 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.181591 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.181974 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.182421 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.182800 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.183141 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.183485 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.183822 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.184756 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.185283 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.185702 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.186168 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.186520 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.186883 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.187230 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.187569 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.187915 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.188275 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.188639 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.189748 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.ssl      = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.190129 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.190623 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.191024 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.191392 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.191805 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_notifications.driver = [] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.192186 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_notifications.retry = -1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.192955 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.193321 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] oslo_messaging_notifications.transport_url = **** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.193810 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] key_manager.backend            = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.194339 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.195188 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.barbican_api_version  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.195704 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.barbican_endpoint     = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.196204 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.barbican_endpoint_type = public {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.196577 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.barbican_region_name  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.196921 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.cafile                = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.197285 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.certfile              = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.197955 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.collect_timing        = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.198314 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.insecure              = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.198656 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.keyfile               = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.199008 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.number_of_retries     = 60 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.199358 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.retry_delay           = 1 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.199704 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.send_service_user_token = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.201240 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.split_loggers         = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.201645 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.timeout               = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.202007 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.verify_ssl            = True {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.202457 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican.verify_ssl_path       = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.202835 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.auth_section = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.203187 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.auth_type = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.203533 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.cafile   = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.203897 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.certfile = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.204249 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.collect_timing = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.206431 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.insecure = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.206743 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.keyfile  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.207102 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.split_loggers = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.207991 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] barbican_service_user.timeout  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.208379 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.approle_role_id          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.208729 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.approle_secret_id        = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.209075 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.cafile                   = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.209719 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.certfile                 = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.210087 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.collect_timing           = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.211121 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.insecure                 = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.211494 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.keyfile                  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.211862 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.kv_mountpoint            = secret {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.212216 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.kv_version               = 2 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.212716 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.namespace                = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.213066 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.root_token_id            = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.213435 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.split_loggers            = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.213781 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.ssl_ca_crt_file          = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.214122 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.timeout                  = None {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.214536 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.use_ssl                  = False {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.214898 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Feb 12 20:52:43.216166 ncdv-master-1 cinder-backup[134160]: DEBUG oslo_service.service [None req-df109739-1b97-4456-a398-87e3f6a82374 None None] ******************************************************************************** {{(pid=134160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}}
Feb 12 20:52:43.232959 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.lockutils [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] Lock "cleanup_incomplete_backups_134160" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.267s {{(pid=134603) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}}
Feb 12 20:52:43.243909 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 20:52:43.253479 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 20:52:43.275275 ncdv-master-1 cinder-backup[134603]: INFO cinder.keymgr.migration [None req-1fa356b6-9fa9-48ef-8768-2bb8333b96b9 None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Feb 12 20:52:43.295447 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.service [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] Creating RPC server for service cinder-backup {{(pid=134603) start /opt/stack/cinder/cinder/service.py:231}}
Feb 12 20:52:43.307683 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.service [None req-87034a10-651a-496d-82d4-c54c1d9b7941 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=134603) start /opt/stack/cinder/cinder/service.py:237}}
Feb 12 20:54:28.344209 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 20:54:28.344940 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 20:55:28.343464 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 20:55:28.344126 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 20:56:28.348831 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 20:56:28.349525 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 20:57:28.353295 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 20:57:28.354014 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 20:58:28.357883 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 20:58:28.358566 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 20:59:28.362372 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 20:59:28.362937 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:00:28.367742 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:00:28.368451 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:01:28.372625 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:01:28.373547 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:02:28.378041 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:02:28.378718 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:03:28.383485 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:03:28.384106 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:04:28.388178 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:04:28.388969 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:05:28.391643 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:05:28.392247 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:06:28.388101 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:06:28.388957 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:07:28.386631 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:07:28.387943 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:08:28.384397 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:08:28.385676 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:09:28.380965 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:09:28.381953 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:10:28.381409 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:10:28.382255 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:11:28.378751 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:11:28.379394 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:12:28.378449 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:12:28.379544 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:13:28.384868 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:13:28.384868 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:14:28.387385 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:14:28.387385 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:15:28.383251 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:15:28.383891 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:16:28.381564 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:16:28.382142 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:17:28.377831 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:17:28.378822 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:18:28.376325 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:18:28.377010 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:19:28.374290 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:19:28.375996 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:20:28.374106 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:20:28.374810 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:21:28.370394 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:21:28.371070 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:22:24.065002 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Create backup started, backup: cd2d6562-6afb-4796-a7ce-726fdb6ebb68 volume: 3d0f4abb-5903-44b8-87d3-462cd29d1ffe snapshot: 0e2af5f2-6f44-4382-996b-59776a0542b3.
Feb 12 21:22:24.156571 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:22:23Z,data_timestamp=2022-02-12T21:22:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-1611843752',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=cd2d6562-6afb-4796-a7ce-726fdb6ebb68,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='07de9e87ea3f495f90baaf85d83b982c',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='0e2af5f2-6f44-4382-996b-59776a0542b3',status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='7190c407ef794576a92dc5616e2ec4e3',volume_id=3d0f4abb-5903-44b8-87d3-462cd29d1ffe)
Feb 12 21:22:26.063086 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:22:26.065830 ncdv-master-1 cinder-backup[134603]: INFO oslo.privsep.daemon [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpsw8rob1o/privsep.sock']
Feb 12 21:22:26.135759 ncdv-master-1 sudo[161144]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpsw8rob1o/privsep.sock
Feb 12 21:22:26.137244 ncdv-master-1 sudo[161144]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 12 21:22:28.375732 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:22:28.376366 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:22:31.533095 ncdv-master-1 sudo[161144]: pam_unix(sudo:session): session closed for user root
Feb 12 21:22:31.540368 ncdv-master-1 cinder-backup[134603]: INFO oslo.privsep.daemon [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Spawned new privsep daemon via rootwrap
Feb 12 21:22:31.541027 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Accepted privsep connection to /tmp/tmpsw8rob1o/privsep.sock {{(pid=134603) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}}
Feb 12 21:22:31.542647 ncdv-master-1 cinder-backup[134603]: INFO oslo.privsep.daemon [-] privsep daemon starting
Feb 12 21:22:31.543138 ncdv-master-1 cinder-backup[134603]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Feb 12 21:22:31.543449 ncdv-master-1 cinder-backup[134603]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Feb 12 21:22:31.543715 ncdv-master-1 cinder-backup[134603]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 161241
Feb 12 21:22:31.545403 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[f155428f-f8e9-4c3c-ba74-85c8ebe2b9f6]: (2,) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.020289 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.050456 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.031s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:22:32.050791 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d1233e1-4964-446d-8278-19ae705007bb]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.052637 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.059719 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[80a577d3-7e26-42ee-8bd0-ec62c92cc3a5]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:22:32.073682 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:22:32.075499 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[80a577d3-7e26-42ee-8bd0-ec62c92cc3a5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.077471 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.095555 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ae0d0fb9-1816-4794-bf2a-d94309252f02]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae0d0fb9-1816-4794-bf2a-d94309252f02]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.104494 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.142639 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.040s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:22:32.143675 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[0520dd14-27a0-4859-8f0e-3f2718d66bac]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.145058 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.231590 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.086s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:22:32.232119 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[c028d460-b344-4909-9eb5-9bef1be29cb6]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.236934 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.271863 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.037s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:22:32.272179 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[651f698e-f656-4c1a-8bbf-924868e002f5]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:22:32.273261 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:22:32.341876 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] CMD "nvme version" returned: 0 in 0.068s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:22:32.352373 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] <== get_connector_properties: return (6287ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:22:33.477232 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:22:33.566673 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:22:33.569506 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39017b8d30>, 'connection_properties': {'name': 'volumes/volume-f155360d-3204-4c16-b0a4-b1db32a53963', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f155360d-3204-4c16-b0a4-b1db32a53963', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:22:33.580911 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:22:33.902411 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] <== connect_volume: return (332ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3901610040>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:22:33.988650 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Starting backup of volume='3d0f4abb-5903-44b8-87d3-462cd29d1ffe'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:22:34.045483 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:22:34.046180 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Creating backup base image='volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.base' for volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe. {{(pid=134603) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}}
Feb 12 21:22:37.226074 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Copying data from volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe. {{(pid=134603) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}}
Feb 12 21:22:37.437003 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferring data between 'volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe' and 'volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.base' {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}}
Feb 12 21:22:37.437402 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] 8 chunks of 134217728 bytes to be transferred {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}}
Feb 12 21:22:50.164275 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 1 of 8 (10300K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:23:01.082970 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 2 of 8 (12004K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:23:13.656437 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 3 of 8 (10425K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:23:25.902798 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 4 of 8 (10703K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:23:28.376398 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:23:28.378118 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:23:37.175518 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 5 of 8 (11628K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:23:49.836812 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 6 of 8 (10352K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:24:00.643633 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 7 of 8 (12129K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:24:12.525871 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Transferred chunk 8 of 8 (11032K/s) {{(pid=134603) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Feb 12 21:24:13.198034 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:24:13.216816 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:24:13.217319 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:24:13.246082 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:24:13.247337 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:24:13.279182 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] No metadata type 'volume-glance-metadata' available {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Feb 12 21:24:13.280017 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Backing up metadata for volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:24:13.583827 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Backup 'cd2d6562-6afb-4796-a7ce-726fdb6ebb68' of volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:24:13.585787 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39017b8d30>, 'connection_properties': {'name': 'volumes/volume-f155360d-3204-4c16-b0a4-b1db32a53963', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f155360d-3204-4c16-b0a4-b1db32a53963', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3901610040>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:24:13.893245 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] <== disconnect_volume: return (306ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:24:14.540321 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-64600ecf-9ea6-4f76-aaba-e9dfb3859fe0 tempest-VolumesSnapshotTestJSON-1959179807 None] Create backup finished. backup: cd2d6562-6afb-4796-a7ce-726fdb6ebb68.
Feb 12 21:24:15.631806 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Delete backup started, backup: cd2d6562-6afb-4796-a7ce-726fdb6ebb68.
Feb 12 21:24:15.634433 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Delete started for backup=cd2d6562-6afb-4796-a7ce-726fdb6ebb68 {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:24:15.634863 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Trying diff format basename='volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.cd2d6562-6afb-4796-a7ce-726fdb6ebb68' for backup base image of volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:24:15.710499 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Image 'volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.cd2d6562-6afb-4796-a7ce-726fdb6ebb68' not found - trying diff format name {{(pid=134603) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}}
Feb 12 21:24:15.970712 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Looking for snapshot of backup base 'volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.base' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:24:15.971089 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Backup base 'volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.base' has no snapshots {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}}
Feb 12 21:24:15.971558 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] No backup snapshot to delete {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Feb 12 21:24:16.158363 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Deleting backup base image='volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.base' of volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe.
Feb 12 21:24:23.493778 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:24:28.376799 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:24:28.377627 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:24:31.887964 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Base backup image='volume-3d0f4abb-5903-44b8-87d3-462cd29d1ffe.backup.base' of volume 3d0f4abb-5903-44b8-87d3-462cd29d1ffe deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:24:32.159555 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Delete of backup 'cd2d6562-6afb-4796-a7ce-726fdb6ebb68' for volume '3d0f4abb-5903-44b8-87d3-462cd29d1ffe' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:24:32.206962 ncdv-master-1 cinder-backup[134603]: WARNING cinder.quota [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Deprecated: Default quota for resource: backups is set by the default quota flag: quota_backups, it is now deprecated. Please use the default quota class for default quota.
Feb 12 21:24:32.208526 ncdv-master-1 cinder-backup[134603]: WARNING cinder.quota [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Deprecated: Default quota for resource: backup_gigabytes is set by the default quota flag: quota_backup_gigabytes, it is now deprecated. Please use the default quota class for default quota.
Feb 12 21:24:32.367654 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Created reservations ['9b7a7a4a-72d8-4f61-9149-bc6d89143884', '0b305c5e-2fdb-403f-ab87-059811de8012'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:24:32.694723 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-281cc085-d10e-42c4-8f02-587c1e5743c2 tempest-VolumesSnapshotTestJSON-1959179807 None] Delete backup finished, backup cd2d6562-6afb-4796-a7ce-726fdb6ebb68 deleted.
Feb 12 21:25:28.381865 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:25:28.381865 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:26:28.387899 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:26:28.389154 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:27:28.387957 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:27:28.388716 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:28:28.385561 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:28:28.387061 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:28:34.815090 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Create backup started, backup: 25c83729-fd36-405d-a329-b96511b4e61b volume: 26ca82ae-203e-4910-8cdd-d0a1425ce5d4.
Feb 12 21:28:34.954095 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:28:34Z,data_timestamp=2022-02-12T21:28:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-2003215460',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=25c83729-fd36-405d-a329-b96511b4e61b,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7a70c35cc73d447eacc7b32dabbae7e3',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='28d91a4daaf74bf88c4a227f8944b7d1',volume_id=26ca82ae-203e-4910-8cdd-d0a1425ce5d4)
Feb 12 21:28:35.009932 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:28:35.014352 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.047516 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.034s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:28:35.047516 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a4e3154-02b9-4ea2-91fb-397a71e4ec8f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:28:35.055478 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.067821 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[5d1b0585-851d-4678-a8a7-403adc518cf1]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d1b0585-851d-4678-a8a7-403adc518cf1]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:28:35.079156 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.099124 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[470220f9-25d4-4e7e-89c5-fb68919c4d56]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:28:35.099466 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:28:35.100776 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[470220f9-25d4-4e7e-89c5-fb68919c4d56]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:28:35.102986 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.138777 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.037s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:28:35.140093 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9a70b91-6746-45ca-bfa7-45ebc3a3387e]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:28:35.144465 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.223214 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.078s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:28:35.223214 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c2eac10-b42a-4ebf-b395-11bd44f4fda8]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:28:35.223214 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.233642 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.011s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:28:35.233971 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb88d73e-4477-40cf-8aba-0c72867f02d6]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:28:35.234767 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:28:35.274209 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] CMD "nvme version" returned: 0 in 0.037s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:28:35.302027 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] <== get_connector_properties: return (270ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:28:36.195221 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:28:36.197067 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39015ba940>, 'connection_properties': {'name': 'volumes/volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '26ca82ae-203e-4910-8cdd-d0a1425ce5d4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:28:36.198301 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:28:36.623538 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] <== connect_volume: return (425ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f390172a880>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:28:36.631520 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:28:36.734517 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Starting backup of volume='26ca82ae-203e-4910-8cdd-d0a1425ce5d4'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:28:36.735860 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Volume file is RBD: attempting optimized backup {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Feb 12 21:28:36.777171 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Creating base image 'volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' {{(pid=134603) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Feb 12 21:28:36.778250 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:28:37.916506 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Using --from-snap 'None' for incremental backup of volume 26ca82ae-203e-4910-8cdd-d0a1425ce5d4. {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Feb 12 21:28:37.918059 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Creating backup snapshot='backup.25c83729-fd36-405d-a329-b96511b4e61b.snap.1644701317.9171085' {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Feb 12 21:28:41.168779 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Performing differential transfer from 'volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4' to 'volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:28:41.168779 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_g8mn0yoz --pool volumes volumes/volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4@backup.25c83729-fd36-405d-a329-b96511b4e61b.snap.1644701317.9171085 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:28:41.170142 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:28:45.162872 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Differential backup transfer completed in 3.9938s {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Feb 12 21:28:45.163598 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:28:45.178030 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:28:45.180996 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:28:45.185840 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:28:45.186303 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:28:45.193538 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] No metadata type 'volume-glance-metadata' available {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Feb 12 21:28:45.194172 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Backing up metadata for volume 26ca82ae-203e-4910-8cdd-d0a1425ce5d4. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:28:45.609655 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Backup '25c83729-fd36-405d-a329-b96511b4e61b' of volume 26ca82ae-203e-4910-8cdd-d0a1425ce5d4 finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:28:45.611355 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39015ba940>, 'connection_properties': {'name': 'volumes/volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '26ca82ae-203e-4910-8cdd-d0a1425ce5d4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f390172a880>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:28:46.552828 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] <== disconnect_volume: return (940ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:28:46.813378 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-7469b4b1-4b9a-4a5d-b2b1-8968b821a647 tempest-VolumesBackupsV39Test-461572751 None] Create backup finished. backup: 25c83729-fd36-405d-a329-b96511b4e61b.
Feb 12 21:28:47.475444 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Delete backup started, backup: 25c83729-fd36-405d-a329-b96511b4e61b.
Feb 12 21:28:47.477645 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Delete started for backup=25c83729-fd36-405d-a329-b96511b4e61b {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:28:47.477645 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Trying diff format basename='volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' for backup base image of volume 26ca82ae-203e-4910-8cdd-d0a1425ce5d4. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:28:47.762323 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Looking for snapshot of backup base 'volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:28:47.763423 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Found snapshot 'backup.25c83729-fd36-405d-a329-b96511b4e61b.snap.1644701317.9171085' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:28:47.763965 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Deleting backup snapshot='backup.25c83729-fd36-405d-a329-b96511b4e61b.snap.1644701317.9171085' {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Feb 12 21:28:51.596407 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Deleting backup base image='volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' of volume 26ca82ae-203e-4910-8cdd-d0a1425ce5d4.
Feb 12 21:28:53.509547 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:28:58.975632 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Base backup image='volume-26ca82ae-203e-4910-8cdd-d0a1425ce5d4.backup.25c83729-fd36-405d-a329-b96511b4e61b' of volume 26ca82ae-203e-4910-8cdd-d0a1425ce5d4 deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:28:59.214605 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Delete of backup '25c83729-fd36-405d-a329-b96511b4e61b' for volume '26ca82ae-203e-4910-8cdd-d0a1425ce5d4' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:28:59.351456 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Created reservations ['964c654d-4f41-45c4-8a92-a5f2d0754b86', '37765fda-aad4-4c55-8a61-03843fb6bd54'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:28:59.656319 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-9913b7ec-e7ad-4aaa-9633-f33a238fc088 tempest-VolumesBackupsV39Test-461572751 None] Delete backup finished, backup 25c83729-fd36-405d-a329-b96511b4e61b deleted.
Feb 12 21:29:28.383120 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:29:28.383784 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:30:28.384128 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:30:28.384128 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:30:59.177121 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Create backup started, backup: 22aaf2aa-508d-4fff-adaa-4bdc119bcb23 volume: 3d099d6f-2ad2-42c9-afe0-4edb1118f461.
Feb 12 21:30:59.255517 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:30:58Z,data_timestamp=2022-02-12T21:30:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-365290829',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=22aaf2aa-508d-4fff-adaa-4bdc119bcb23,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='4fb7edbcd0864f13be6f5138d1350e12',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='d083b81672cd463fb6cce6f8a602bb4f',volume_id=3d099d6f-2ad2-42c9-afe0-4edb1118f461)
Feb 12 21:30:59.309395 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:30:59.311281 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.321130 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:30:59.321751 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[830eeddf-ae5e-447c-b9aa-903f1fea1c8e]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:30:59.323248 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.329999 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[609d8ad1-10dd-46bc-ae76-7ea5700770f9]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:30:59.330611 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:30:59.332452 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[609d8ad1-10dd-46bc-ae76-7ea5700770f9]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:30:59.335606 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.341943 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e911f253-620a-4799-8e49-bc9f9cdd0fb2]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:30:59.342529 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:30:59.343680 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[e911f253-620a-4799-8e49-bc9f9cdd0fb2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:30:59.345218 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.366756 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.021s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:30:59.367940 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[50e36894-61dc-4f09-8b20-a20caa36461b]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:30:59.370795 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.392915 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.021s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:30:59.393201 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[374b9f4f-5d2b-45c5-bea9-95ac6bf0cd6e]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:30:59.396749 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.412669 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.016s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:30:59.413175 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[483fc95f-9994-4dff-99d7-ce4cfb2e30e6]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:30:59.415235 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:30:59.456048 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] CMD "nvme version" returned: 0 in 0.041s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:30:59.457417 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] <== get_connector_properties: return (147ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:31:00.759753 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:31:00.762645 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3901867d00>, 'connection_properties': {'name': 'volumes/volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '3d099d6f-2ad2-42c9-afe0-4edb1118f461', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:31:00.764202 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:31:00.960044 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] <== connect_volume: return (196ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c6fe50>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:31:00.968861 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:31:01.079610 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Starting backup of volume='3d099d6f-2ad2-42c9-afe0-4edb1118f461'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:31:01.080705 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Volume file is RBD: attempting optimized backup {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Feb 12 21:31:01.127103 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Creating base image 'volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' {{(pid=134603) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Feb 12 21:31:01.127801 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:31:02.296924 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Using --from-snap 'None' for incremental backup of volume 3d099d6f-2ad2-42c9-afe0-4edb1118f461. {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Feb 12 21:31:02.297533 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Creating backup snapshot='backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23.snap.1644701462.296952' {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Feb 12 21:31:04.712800 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Performing differential transfer from 'volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461' to 'volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:31:04.713957 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_os74o3v4 --pool volumes volumes/volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461@backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23.snap.1644701462.296952 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:31:04.714441 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:31:07.685579 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Differential backup transfer completed in 2.9735s {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Feb 12 21:31:07.686374 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:31:07.702447 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:31:07.702447 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:31:07.710039 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:31:07.710430 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:31:07.718266 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] No metadata type 'volume-glance-metadata' available {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Feb 12 21:31:07.718841 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Backing up metadata for volume 3d099d6f-2ad2-42c9-afe0-4edb1118f461. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:31:07.918105 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Backup '22aaf2aa-508d-4fff-adaa-4bdc119bcb23' of volume 3d099d6f-2ad2-42c9-afe0-4edb1118f461 finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:31:07.921247 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3901867d00>, 'connection_properties': {'name': 'volumes/volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '3d099d6f-2ad2-42c9-afe0-4edb1118f461', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c6fe50>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:31:08.146360 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] <== disconnect_volume: return (224ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:31:08.468038 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-cee15500-3c36-47cb-8e31-5bad7b415566 tempest-VolumesBackupsTest-835000994 None] Create backup finished. backup: 22aaf2aa-508d-4fff-adaa-4bdc119bcb23.
Feb 12 21:31:09.624542 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Delete backup started, backup: 22aaf2aa-508d-4fff-adaa-4bdc119bcb23.
Feb 12 21:31:09.626948 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Delete started for backup=22aaf2aa-508d-4fff-adaa-4bdc119bcb23 {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:31:09.627372 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Trying diff format basename='volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' for backup base image of volume 3d099d6f-2ad2-42c9-afe0-4edb1118f461. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:31:10.029996 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Looking for snapshot of backup base 'volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:31:10.030806 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Found snapshot 'backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23.snap.1644701462.296952' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:31:10.031403 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Deleting backup snapshot='backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23.snap.1644701462.296952' {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Feb 12 21:31:13.541062 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:31:13.898637 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Deleting backup base image='volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' of volume 3d099d6f-2ad2-42c9-afe0-4edb1118f461.
Feb 12 21:31:21.756259 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Base backup image='volume-3d099d6f-2ad2-42c9-afe0-4edb1118f461.backup.22aaf2aa-508d-4fff-adaa-4bdc119bcb23' of volume 3d099d6f-2ad2-42c9-afe0-4edb1118f461 deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:31:22.050225 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Delete of backup '22aaf2aa-508d-4fff-adaa-4bdc119bcb23' for volume '3d099d6f-2ad2-42c9-afe0-4edb1118f461' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:31:22.153249 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Created reservations ['1e2e7e21-7989-4a2e-9365-d972fd2f3f14', 'b6dbc1ab-329e-4475-9664-13780120cce3'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:31:22.363621 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-4d7bbdf3-13ba-40ad-aa92-25a3ca9765cb tempest-VolumesBackupsTest-835000994 None] Delete backup finished, backup 22aaf2aa-508d-4fff-adaa-4bdc119bcb23 deleted.
Feb 12 21:31:28.383730 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:31:28.384333 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:31:56.213278 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Create backup started, backup: 91a00afe-73f8-4fb5-b868-f5cd70a56fc7 volume: 8634339a-63d4-437c-aab7-c9cd3446b531.
Feb 12 21:31:56.320412 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:31:55Z,data_timestamp=2022-02-12T21:31:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1073773826',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=91a00afe-73f8-4fb5-b868-f5cd70a56fc7,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='4fb7edbcd0864f13be6f5138d1350e12',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='d083b81672cd463fb6cce6f8a602bb4f',volume_id=8634339a-63d4-437c-aab7-c9cd3446b531)
Feb 12 21:31:56.408675 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:31:56.414369 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.426190 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:31:56.426526 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[10ef288a-d812-4ae5-ad8a-fb90373038b9]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:31:56.429709 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.437648 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4e63295e-aa57-4722-8002-67e8cfc904c4]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:31:56.439032 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:31:56.441255 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e63295e-aa57-4722-8002-67e8cfc904c4]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:31:56.447374 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.457234 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3f926dd4-723d-4c17-9f02-3850523c9646]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:31:56.457916 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:31:56.459574 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f926dd4-723d-4c17-9f02-3850523c9646]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:31:56.461253 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.476460 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.013s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:31:56.476901 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[5dbf1004-4d4d-4f2b-8936-8fb56fc2ecfe]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:31:56.478976 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.495573 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.016s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:31:56.496173 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b8fb754-75a8-4c49-85b7-e8b1c8e31e77]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:31:56.498049 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.507232 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:31:56.508111 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[36a8638b-de68-4106-8074-1b06a29d29c4]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:31:56.509732 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:31:56.544191 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] CMD "nvme version" returned: 0 in 0.034s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:31:56.546275 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:31:57.904618 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:31:57.906145 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3901582970>, 'connection_properties': {'name': 'volumes/volume-8634339a-63d4-437c-aab7-c9cd3446b531', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8634339a-63d4-437c-aab7-c9cd3446b531', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:31:57.907252 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:31:58.319283 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] <== connect_volume: return (412ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c6e760>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:31:58.327792 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:31:58.458475 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Starting backup of volume='8634339a-63d4-437c-aab7-c9cd3446b531'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:31:58.460566 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Volume file is RBD: attempting optimized backup {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Feb 12 21:31:58.502085 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Creating base image 'volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' {{(pid=134603) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Feb 12 21:31:58.502860 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:31:59.742296 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Using --from-snap 'None' for incremental backup of volume 8634339a-63d4-437c-aab7-c9cd3446b531. {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Feb 12 21:31:59.742929 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Creating backup snapshot='backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203' {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Feb 12 21:32:02.396654 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Performing differential transfer from 'volume-8634339a-63d4-437c-aab7-c9cd3446b531' to 'volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:32:02.397413 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_6fgzmgdb --pool volumes volumes/volume-8634339a-63d4-437c-aab7-c9cd3446b531@backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:32:02.398167 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:32:13.687684 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Create backup started, backup: 278e59a9-78b0-48d1-ab85-d144c12c2de5 volume: b3511a55-12b4-476c-b22f-f927dc31e59a.
Feb 12 21:32:13.831314 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:32:13Z,data_timestamp=2022-02-12T21:32:13Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=278e59a9-78b0-48d1-ab85-d144c12c2de5,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='cfcb9d238a52483a94553e900468a87c',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='dd1a01a1f86a4f4ca3997db04ed1b4f8',volume_id=b3511a55-12b4-476c-b22f-f927dc31e59a)
Feb 12 21:32:13.878465 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:32:13.880130 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:13.893939 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:32:13.894811 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[c294a5f7-42df-484f-8fe8-d60192936d1b]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:32:13.900821 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:13.914154 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4f2c6433-719c-4fcc-be32-38b0d6e3cf39]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:32:13.914878 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:32:13.916421 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f2c6433-719c-4fcc-be32-38b0d6e3cf39]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:32:13.920184 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:13.928795 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e1b7c23d-9953-4e88-b598-325b1be22026]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:32:13.929436 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:32:13.932179 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1b7c23d-9953-4e88-b598-325b1be22026]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:32:13.936050 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:13.947649 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.013s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:32:13.948215 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[0835af1b-59c7-4e75-9711-9365e59662e2]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:32:13.953154 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:13.963945 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.012s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:32:13.964312 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[3373b3f5-582c-4ca9-9c81-5eb6437a8523]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:32:13.968223 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:13.980410 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.014s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:32:13.981290 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[b528ad34-24e0-475d-8f2c-7e58dea3acdb]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:32:13.982293 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:32:14.018370 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] CMD "nvme version" returned: 0 in 0.036s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:32:14.019647 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] <== get_connector_properties: return (140ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:32:14.896961 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:32:14.896961 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39017be8e0>, 'connection_properties': {'name': 'volumes/volume-b3511a55-12b4-476c-b22f-f927dc31e59a', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b3511a55-12b4-476c-b22f-f927dc31e59a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:32:14.897654 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:32:15.396711 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] <== connect_volume: return (500ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c3abb0>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:32:15.409419 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:32:15.632360 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Starting backup of volume='b3511a55-12b4-476c-b22f-f927dc31e59a'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:32:15.633261 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Volume file is RBD: attempting optimized backup {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Feb 12 21:32:15.654350 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Creating base image 'volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' {{(pid=134603) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Feb 12 21:32:15.654749 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:32:17.563104 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Using --from-snap 'None' for incremental backup of volume b3511a55-12b4-476c-b22f-f927dc31e59a. {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Feb 12 21:32:17.563797 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Creating backup snapshot='backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808' {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Feb 12 21:32:21.736018 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Performing differential transfer from 'volume-b3511a55-12b4-476c-b22f-f927dc31e59a' to 'volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:32:21.736661 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_fst1_cfw --pool volumes volumes/volume-b3511a55-12b4-476c-b22f-f927dc31e59a@backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:32:21.736801 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:33:13.330582 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Differential backup transfer completed in 70.9330s {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Feb 12 21:33:13.332269 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:33:13.350643 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:33:13.355659 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:33:13.357624 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:33:13.358007 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:33:13.366747 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Feb 12 21:33:13.367220 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Backing up metadata for volume 8634339a-63d4-437c-aab7-c9cd3446b531. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:33:13.906070 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Backup '91a00afe-73f8-4fb5-b868-f5cd70a56fc7' of volume 8634339a-63d4-437c-aab7-c9cd3446b531 finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:33:13.908665 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3901582970>, 'connection_properties': {'name': 'volumes/volume-8634339a-63d4-437c-aab7-c9cd3446b531', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8634339a-63d4-437c-aab7-c9cd3446b531', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c6e760>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:14.580830 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] <== disconnect_volume: return (671ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:14.811786 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-21a70fde-e83b-434f-aad7-58124853c1fc tempest-VolumesBackupsTest-835000994 None] Create backup finished. backup: 91a00afe-73f8-4fb5-b868-f5cd70a56fc7.
Feb 12 21:33:19.245611 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Restore backup started, backup: 91a00afe-73f8-4fb5-b868-f5cd70a56fc7 volume: 3445accb-08f9-4e70-b9c5-49b4639a3aa1.
Feb 12 21:33:19.262154 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:19.266914 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.303363 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.040s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:19.303705 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[30bba69e-4fa9-45fd-8ef8-17bcc6c60e51]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:19.305713 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.322503 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[597da0e5-70f6-4bbf-ae1a-b2a20a449d18]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:33:19.322840 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:33:19.323860 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[597da0e5-70f6-4bbf-ae1a-b2a20a449d18]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:19.325058 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.346599 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[9871f7e2-0f13-465d-a278-f58323ed5123]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:33:19.346903 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:33:19.350097 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[9871f7e2-0f13-465d-a278-f58323ed5123]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:19.350097 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.376200 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.026s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:19.379085 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[62ffb1f8-d8d9-4598-bc30-c6401dbbc288]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:19.384622 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.400111 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.016s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:19.400406 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[305ce5a7-a56d-4bff-90ba-24b4af6b6808]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:19.404855 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.449785 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.047s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:19.450319 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e2b6471-6107-4fc0-b4b3-e7016a1e6c4a]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:19.451331 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:19.503693 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] CMD "nvme version" returned: 0 in 0.052s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:19.504911 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] <== get_connector_properties: return (241ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:20.441924 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:33:20.443512 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39017a59d0>, 'connection_properties': {'name': 'volumes/volume-3445accb-08f9-4e70-b9c5-49b4639a3aa1', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '3445accb-08f9-4e70-b9c5-49b4639a3aa1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:20.445103 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:33:20.834745 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] <== connect_volume: return (390ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900be2430>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:20.842729 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:33:20.856084 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Starting restore from Ceph backup=91a00afe-73f8-4fb5-b868-f5cd70a56fc7 to volume=volume-3445accb-08f9-4e70-b9c5-49b4639a3aa1 {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Feb 12 21:33:21.039494 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Looking for snapshot of backup base 'volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:33:21.040302 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Found snapshot 'backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:33:21.051547 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Volume file is RBD. {{(pid=134603) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Feb 12 21:33:21.108370 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Attempting differential restore. {{(pid=134603) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Feb 12 21:33:21.108853 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Attempting incremental restore from base='volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' snap='backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203' {{(pid=134603) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Feb 12 21:33:21.109277 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Performing differential transfer from 'volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' to 'volume-3445accb-08f9-4e70-b9c5-49b4639a3aa1' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:33:21.109715 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7@backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:33:21.110082 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_l3ck1xul --pool volumes - volumes/volume-3445accb-08f9-4e70-b9c5-49b4639a3aa1' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:33:24.777187 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Differential backup transfer completed in 63.0398s {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Feb 12 21:33:24.779342 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:33:24.811871 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:33:24.815866 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:33:24.831897 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:33:24.831897 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:33:24.839951 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Feb 12 21:33:24.840548 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Backing up metadata for volume b3511a55-12b4-476c-b22f-f927dc31e59a. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:33:25.367944 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Backup '278e59a9-78b0-48d1-ab85-d144c12c2de5' of volume b3511a55-12b4-476c-b22f-f927dc31e59a finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:33:25.367944 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39017be8e0>, 'connection_properties': {'name': 'volumes/volume-b3511a55-12b4-476c-b22f-f927dc31e59a', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b3511a55-12b4-476c-b22f-f927dc31e59a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c3abb0>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:26.554369 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] <== disconnect_volume: return (1189ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:26.764658 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-9d1c8fa6-3edb-4a62-b71e-9621cf52e64d tempest-TestVolumeBackupRestore-1020972158 None] Create backup finished. backup: 278e59a9-78b0-48d1-ab85-d144c12c2de5.
Feb 12 21:33:28.382603 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:33:28.384042 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:33:31.674023 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Restore backup started, backup: 278e59a9-78b0-48d1-ab85-d144c12c2de5 volume: 29b7e6aa-158b-4248-a1b3-80e64eeff62a.
Feb 12 21:33:31.692509 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:31.702525 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.726753 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.031s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:31.727283 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[66173e6f-13e3-4369-8db8-07aca8f87520]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:31.731721 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.749411 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e421bd6c-35a5-4fcf-8a50-a7976b1e5377]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:33:31.750891 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:33:31.752344 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[e421bd6c-35a5-4fcf-8a50-a7976b1e5377]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:31.760402 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.781943 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c4db8921-140a-4ef1-a789-2c4e17876de8]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:33:31.782293 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:33:31.783347 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4db8921-140a-4ef1-a789-2c4e17876de8]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:31.791397 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.810335 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.017s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:31.810624 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9a681e7-a13e-472f-a26f-63db1985df2e]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:31.815454 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.850021 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.033s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:31.850365 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[959d9427-47f3-4b76-9a26-dbf28d5e64d1]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:31.853874 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.884487 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.030s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:31.884971 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[935895de-0f3c-4f75-b20e-df21d8adce75]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:31.886531 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:31.936978 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] CMD "nvme version" returned: 0 in 0.050s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:31.940031 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] <== get_connector_properties: return (246ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:33.023984 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:33:33.026222 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3900c69190>, 'connection_properties': {'name': 'volumes/volume-29b7e6aa-158b-4248-a1b3-80e64eeff62a', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '29b7e6aa-158b-4248-a1b3-80e64eeff62a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:33.027883 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:33:33.337021 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] <== connect_volume: return (309ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3901713160>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:33.359801 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:33:33.377373 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Starting restore from Ceph backup=278e59a9-78b0-48d1-ab85-d144c12c2de5 to volume=volume-29b7e6aa-158b-4248-a1b3-80e64eeff62a {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Feb 12 21:33:33.843799 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Looking for snapshot of backup base 'volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:33:33.844563 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Found snapshot 'backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:33:33.857699 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Volume file is RBD. {{(pid=134603) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Feb 12 21:33:34.008545 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Attempting differential restore. {{(pid=134603) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Feb 12 21:33:34.009035 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Attempting incremental restore from base='volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' snap='backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808' {{(pid=134603) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Feb 12 21:33:34.009537 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Performing differential transfer from 'volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' to 'volume-29b7e6aa-158b-4248-a1b3-80e64eeff62a' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:33:34.009936 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5@backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:33:34.010299 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_86b1zhdt --pool volumes - volumes/volume-29b7e6aa-158b-4248-a1b3-80e64eeff62a' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:33:48.241336 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Create backup started, backup: 0ee17d6a-58ae-4bdd-9100-172dbd9af093 volume: a0daa52e-7dba-4527-9c5a-cc39ae607708.
Feb 12 21:33:48.325502 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:33:47Z,data_timestamp=2022-02-12T21:33:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-368492704',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=0ee17d6a-58ae-4bdd-9100-172dbd9af093,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='4d6c242b7ff346f5b5cb6b75c4321618',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='c5e0e7a6b76b4cfeb2c032cfff336d9e',volume_id=a0daa52e-7dba-4527-9c5a-cc39ae607708)
Feb 12 21:33:48.380264 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:48.381653 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.393209 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:48.393576 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[de28b184-a2e3-4712-b612-e300b75c750e]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:48.398138 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.404726 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[eece693c-e34f-41e5-8a52-0c0efa7f3d91]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:33:48.405909 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:33:48.407341 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[eece693c-e34f-41e5-8a52-0c0efa7f3d91]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:48.408552 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.421926 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4dcc12d2-2a48-4c53-bc2b-98627a02209e]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:33:48.422478 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:33:48.423467 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[4dcc12d2-2a48-4c53-bc2b-98627a02209e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:48.424644 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.438923 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:48.439902 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[811fca9e-c32c-4dba-938c-8330deaea0dc]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:48.441651 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.454104 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.012s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:48.454417 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[636eb6be-67f8-4fb4-a5bd-4c4b991eff56]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:48.456021 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.467633 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.011s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:48.467998 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[a991b2f2-12cc-48ee-8ca0-aba20f5d4d8d]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:33:48.468785 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:33:48.514007 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] CMD "nvme version" returned: 0 in 0.045s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:33:48.515121 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] <== get_connector_properties: return (134ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:49.297377 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:33:49.298906 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3901531ee0>, 'connection_properties': {'name': 'volumes/volume-a0daa52e-7dba-4527-9c5a-cc39ae607708', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a0daa52e-7dba-4527-9c5a-cc39ae607708', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:33:49.300331 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:33:50.742445 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] <== connect_volume: return (1442ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3901633190>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:33:50.750685 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:33:50.839903 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Starting backup of volume='a0daa52e-7dba-4527-9c5a-cc39ae607708'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:33:50.841095 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Volume file is RBD: attempting optimized backup {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Feb 12 21:33:50.875116 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Creating base image 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' {{(pid=134603) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Feb 12 21:33:50.875524 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:33:53.639487 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Using --from-snap 'None' for incremental backup of volume a0daa52e-7dba-4527-9c5a-cc39ae607708. {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Feb 12 21:33:53.640991 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Creating backup snapshot='backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557' {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Feb 12 21:33:57.881624 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Performing differential transfer from 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708' to 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:33:57.882359 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd__3ted8gv --pool volumes volumes/volume-a0daa52e-7dba-4527-9c5a-cc39ae607708@backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:33:57.882769 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:34:14.051603 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Differential backup transfer completed in 16.1693s {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Feb 12 21:34:14.052368 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:34:14.067260 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:34:14.068005 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:34:14.075781 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:34:14.076506 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:34:14.084088 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] No metadata type 'volume-glance-metadata' available {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Feb 12 21:34:14.084709 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Backing up metadata for volume a0daa52e-7dba-4527-9c5a-cc39ae607708. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:34:14.776230 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Backup '0ee17d6a-58ae-4bdd-9100-172dbd9af093' of volume a0daa52e-7dba-4527-9c5a-cc39ae607708 finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:34:14.778141 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3901531ee0>, 'connection_properties': {'name': 'volumes/volume-a0daa52e-7dba-4527-9c5a-cc39ae607708', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a0daa52e-7dba-4527-9c5a-cc39ae607708', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3901633190>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:15.609747 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] <== disconnect_volume: return (830ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:15.801396 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-1443d1c9-24da-4515-8de6-33614bf22a14 tempest-VolumesBackupsAdminTest-354663248 None] Create backup finished. backup: 0ee17d6a-58ae-4bdd-9100-172dbd9af093.
Feb 12 21:34:16.190029 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-6e440fce-ce2b-49d9-a306-db3dc761cbfb tempest-VolumesBackupsAdminTest-1031776151 None] Export record started, backup: 0ee17d6a-58ae-4bdd-9100-172dbd9af093.
Feb 12 21:34:16.191851 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-6e440fce-ce2b-49d9-a306-db3dc761cbfb tempest-VolumesBackupsAdminTest-1031776151 None] Export record finished, backup 0ee17d6a-58ae-4bdd-9100-172dbd9af093 exported.
Feb 12 21:34:16.650433 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-c703eee8-5bb2-457c-b59e-d92a2247b782 tempest-VolumesBackupsAdminTest-1031776151 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiZmRiMDhjNTMtMWE5OC00YWNlLTlkMzMtZjg4NDY1NTQ3YTllIiwgInVzZXJfaWQiOiAiYzVlMGU3YTZiNzZiNGNmZWIyYzAzMmNmZmYzMzZkOWUiLCAicHJvamVjdF9pZCI6ICI0ZDZjMjQyYjdmZjM0NmY1YjVjYjZiNzVjNDMyMTYxOCIsICJ2b2x1bWVfaWQiOiAiYTBkYWE1MmUtN2RiYS00NTI3LTljNWEtY2MzOWFlNjA3NzA4IiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC0zNjg0OTI3MDQiLCAiZGlzcGxheV9kZXNjcmlwdGlvbiI6IG51bGwsICJzZXJ2aWNlX21ldGFkYXRhIjogIntcImJhc2VcIjogXCJ2b2x1bWUtYTBkYWE1MmUtN2RiYS00NTI3LTljNWEtY2MzOWFlNjA3NzA4LmJhY2t1cC4wZWUxN2Q2YS01OGFlLTRiZGQtOTEwMC0xNzJkYmQ5YWYwOTNcIn0iLCAic2VydmljZSI6ICJjaW5kZXIuYmFja3VwLmRyaXZlcnMuY2VwaC5DZXBoQmFja3VwRHJpdmVyIiwgIm9iamVjdF9jb3VudCI6IDAsICJ0ZW1wX3ZvbHVtZV9pZCI6IG51bGwsICJ0ZW1wX3NuYXBzaG90X2lkIjogbnVsbCwgIm51bV9kZXBlbmRlbnRfYmFja3VwcyI6IDAsICJzbmFwc2hvdF9pZCI6IG51bGwsICJkYXRhX3RpbWVzdGFtcCI6ICIyMDIyLTAyLTEyVDIxOjMzOjQ4WiIsICJyZXN0b3JlX3ZvbHVtZV9pZCI6IG51bGwsICJtZXRhZGF0YSI6IHt9LCAiZW5jcnlwdGlvbl9rZXlfaWQiOiBudWxsLCAiY3JlYXRlZF9hdCI6ICIyMDIyLTAyLTEyVDIxOjMzOjQ4WiIsICJ1cGRhdGVkX2F0IjogIjIwMjItMDItMTJUMjE6MzQ6MTZaIiwgImRlbGV0ZWRfYXQiOiBudWxsLCAiZGVsZXRlZCI6IGZhbHNlfQ==.
Feb 12 21:34:16.658632 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-c703eee8-5bb2-457c-b59e-d92a2247b782 tempest-VolumesBackupsAdminTest-1031776151 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:34:16.878887 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-c703eee8-5bb2-457c-b59e-d92a2247b782 tempest-VolumesBackupsAdminTest-1031776151 None] Import record id fdb08c53-1a98-4ace-9d33-f88465547a9e metadata from driver finished.
Feb 12 21:34:28.388323 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:34:28.391817 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:34:30.742234 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Restore backup started, backup: 0ee17d6a-58ae-4bdd-9100-172dbd9af093 volume: 542debdc-b1de-48aa-8a7e-2e6fbb4522d7.
Feb 12 21:34:30.779348 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:30.790999 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.826910 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.040s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:30.828125 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[35c02cbb-4bb9-492f-bf6b-d489bc22fceb]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:30.832545 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.842235 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[634b7b71-db01-444a-bd0e-444105203199]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:34:30.842990 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:34:30.844040 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[634b7b71-db01-444a-bd0e-444105203199]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:30.844970 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.856223 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3d481ecc-a438-404a-98f3-ea93cdaca7c4]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:34:30.856512 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:34:30.858602 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d481ecc-a438-404a-98f3-ea93cdaca7c4]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:30.860183 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.873399 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.013s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:30.873720 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3c7d8c3-8c57-4362-b3cd-cb1c7f26d2b4]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:30.877763 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.895406 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.020s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:30.895616 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[007ae59f-4498-4ca4-bf12-360f7a4e006d]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:30.898979 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.907608 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.010s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:30.907943 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3a2b685-9fbb-4431-94d7-b804074a3a4c]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:30.908756 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:30.950422 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] CMD "nvme version" returned: 0 in 0.041s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:30.952450 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] <== get_connector_properties: return (171ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:31.771107 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:34:31.772674 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3900bf8280>, 'connection_properties': {'name': 'volumes/volume-542debdc-b1de-48aa-8a7e-2e6fbb4522d7', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '542debdc-b1de-48aa-8a7e-2e6fbb4522d7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:31.773804 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:34:32.399372 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] <== connect_volume: return (626ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900be25b0>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:32.419539 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:34:32.442779 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Starting restore from Ceph backup=0ee17d6a-58ae-4bdd-9100-172dbd9af093 to volume=volume-542debdc-b1de-48aa-8a7e-2e6fbb4522d7 {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Feb 12 21:34:32.477823 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Restore transfer completed in 71.3674s {{(pid=134603) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Feb 12 21:34:32.479456 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Feb 12 21:34:32.561228 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Restoring volume base metadata {{(pid=134603) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Feb 12 21:34:32.563695 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Looking for snapshot of backup base 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:34:32.565741 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Found snapshot 'backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:34:32.583869 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Volume file is RBD. {{(pid=134603) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Feb 12 21:34:32.694212 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Attempting differential restore. {{(pid=134603) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Feb 12 21:34:32.694855 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Attempting incremental restore from base='volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' snap='backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557' {{(pid=134603) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Feb 12 21:34:32.695339 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Performing differential transfer from 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' to 'volume-542debdc-b1de-48aa-8a7e-2e6fbb4522d7' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:34:32.696001 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093@backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:34:32.696459 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_141q3iu3 --pool volumes - volumes/volume-542debdc-b1de-48aa-8a7e-2e6fbb4522d7' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:34:32.764448 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] No metadata of type 'volume-metadata' to restore {{(pid=134603) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Feb 12 21:34:32.764871 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Restoring volume glance metadata {{(pid=134603) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Feb 12 21:34:34.067314 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Restore to volume 3445accb-08f9-4e70-b9c5-49b4639a3aa1 finished successfully. {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Feb 12 21:34:34.072498 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f39017a59d0>, 'connection_properties': {'name': 'volumes/volume-3445accb-08f9-4e70-b9c5-49b4639a3aa1', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '3445accb-08f9-4e70-b9c5-49b4639a3aa1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900be2430>}, 'force': True, ': False}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:34.218510 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] <== disconnect_volume: return (148ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:34.283919 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.manager [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Encryption key ID for volume 3445accb-08f9-4e70-b9c5-49b4639a3aa1 already matches encryption key ID in backup 91a00afe-73f8-4fb5-b868-f5cd70a56fc7. {{(pid=134603) _run_restore /opt/stack/cinder/cinder/backup/manager.py:824}}
Feb 12 21:34:34.564522 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-8d82f7fb-8533-4d5e-bc9f-ef092143d1fb tempest-VolumesBackupsTest-835000994 None] Finished restoring backup 91a00afe-73f8-4fb5-b868-f5cd70a56fc7 to volume 3445accb-08f9-4e70-b9c5-49b4639a3aa1.
Feb 12 21:34:36.629952 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Restore transfer completed in 3.9344s {{(pid=134603) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Feb 12 21:34:36.632234 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Feb 12 21:34:36.679451 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Restoring volume base metadata {{(pid=134603) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Feb 12 21:34:36.752484 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] No metadata of type 'volume-metadata' to restore {{(pid=134603) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Feb 12 21:34:36.753968 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=134603) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Feb 12 21:34:36.758593 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Restore to volume 542debdc-b1de-48aa-8a7e-2e6fbb4522d7 finished successfully. {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Feb 12 21:34:36.762217 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3900bf8280>, 'connection_properties': {'name': 'volumes/volume-542debdc-b1de-48aa-8a7e-2e6fbb4522d7', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '542debdc-b1de-48aa-8a7e-2e6fbb4522d7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900be25b0>}, 'force': True, ': False}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:36.940629 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] <== disconnect_volume: return (177ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:38.783037 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.manager [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Encryption key ID for volume 542debdc-b1de-48aa-8a7e-2e6fbb4522d7 already matches encryption key ID in backup 0ee17d6a-58ae-4bdd-9100-172dbd9af093. {{(pid=134603) _run_restore /opt/stack/cinder/cinder/backup/manager.py:824}}
Feb 12 21:34:39.217019 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-04f84363-5457-4805-b757-95d34a52e2c6 tempest-VolumesBackupsAdminTest-354663248 None] Finished restoring backup 0ee17d6a-58ae-4bdd-9100-172dbd9af093 to volume 542debdc-b1de-48aa-8a7e-2e6fbb4522d7.
Feb 12 21:34:39.430575 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Restore transfer completed in 65.4190s {{(pid=134603) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Feb 12 21:34:39.430575 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Feb 12 21:34:39.537517 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Restoring volume base metadata {{(pid=134603) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Feb 12 21:34:39.644878 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] No metadata of type 'volume-metadata' to restore {{(pid=134603) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Feb 12 21:34:39.645247 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Restoring volume glance metadata {{(pid=134603) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Feb 12 21:34:40.948054 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Restore to volume 29b7e6aa-158b-4248-a1b3-80e64eeff62a finished successfully. {{(pid=134603) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Feb 12 21:34:40.953897 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3900c69190>, 'connection_properties': {'name': 'volumes/volume-29b7e6aa-158b-4248-a1b3-80e64eeff62a', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '29b7e6aa-158b-4248-a1b3-80e64eeff62a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3901713160>}, 'force': True, ': False}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:41.152652 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] <== disconnect_volume: return (200ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:41.168959 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Delete backup started, backup: fdb08c53-1a98-4ace-9d33-f88465547a9e.
Feb 12 21:34:41.176522 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Delete started for backup=fdb08c53-1a98-4ace-9d33-f88465547a9e {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:34:41.178315 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Trying diff format basename='volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' for backup base image of volume a0daa52e-7dba-4527-9c5a-cc39ae607708. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:34:41.620440 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Looking for snapshot of backup base 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:34:41.621285 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Backup 'fdb08c53-1a98-4ace-9d33-f88465547a9e' has no snapshot {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}}
Feb 12 21:34:41.621940 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] No backup snapshot to delete {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Feb 12 21:34:41.867074 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Backup base image of volume a0daa52e-7dba-4527-9c5a-cc39ae607708 still has 1 snapshots so skipping base image delete.
Feb 12 21:34:42.062288 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Metadata backup object 'backup.fdb08c53-1a98-4ace-9d33-f88465547a9e.meta' not found - ignoring {{(pid=134603) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}}
Feb 12 21:34:42.070473 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Delete of backup 'fdb08c53-1a98-4ace-9d33-f88465547a9e' for volume 'a0daa52e-7dba-4527-9c5a-cc39ae607708' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:34:42.078482 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:34:42.216963 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Created reservations ['fdd51b4a-f12e-4a72-958a-667cd018654b', 'b9fccf98-5d90-47d8-ab86-00db71d1a961'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:34:42.445188 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-547ac7a0-5dec-4b9a-98f4-51d3d8d04ac2 tempest-VolumesBackupsAdminTest-1031776151 None] Delete backup finished, backup fdb08c53-1a98-4ace-9d33-f88465547a9e deleted.
Feb 12 21:34:43.392630 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Delete backup started, backup: 0ee17d6a-58ae-4bdd-9100-172dbd9af093.
Feb 12 21:34:43.402140 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Delete started for backup=0ee17d6a-58ae-4bdd-9100-172dbd9af093 {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:34:43.402747 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Trying diff format basename='volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' for backup base image of volume a0daa52e-7dba-4527-9c5a-cc39ae607708. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:34:43.715700 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:34:43.721068 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Looking for snapshot of backup base 'volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:34:43.721583 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Found snapshot 'backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:34:43.721956 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Deleting backup snapshot='backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093.snap.1644701633.640557' {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Feb 12 21:34:44.590459 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.manager [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Encryption key ID for volume 29b7e6aa-158b-4248-a1b3-80e64eeff62a already matches encryption key ID in backup 278e59a9-78b0-48d1-ab85-d144c12c2de5. {{(pid=134603) _run_restore /opt/stack/cinder/cinder/backup/manager.py:824}}
Feb 12 21:34:44.896772 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-2b5487ba-cbd0-4931-b501-3893a3ad874e tempest-TestVolumeBackupRestore-1020972158 None] Finished restoring backup 278e59a9-78b0-48d1-ab85-d144c12c2de5 to volume 29b7e6aa-158b-4248-a1b3-80e64eeff62a.
Feb 12 21:34:47.347941 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Deleting backup base image='volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' of volume a0daa52e-7dba-4527-9c5a-cc39ae607708.
Feb 12 21:34:49.261839 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Delete backup started, backup: 91a00afe-73f8-4fb5-b868-f5cd70a56fc7.
Feb 12 21:34:49.263269 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Delete started for backup=91a00afe-73f8-4fb5-b868-f5cd70a56fc7 {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:34:49.263784 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Trying diff format basename='volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' for backup base image of volume 8634339a-63d4-437c-aab7-c9cd3446b531. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:34:49.592939 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Looking for snapshot of backup base 'volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:34:49.595320 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Found snapshot 'backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:34:49.604830 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Deleting backup snapshot='backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7.snap.1644701519.7423203' {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Feb 12 21:34:53.234083 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Deleting backup base image='volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' of volume 8634339a-63d4-437c-aab7-c9cd3446b531.
Feb 12 21:34:53.519783 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Base backup image='volume-a0daa52e-7dba-4527-9c5a-cc39ae607708.backup.0ee17d6a-58ae-4bdd-9100-172dbd9af093' of volume a0daa52e-7dba-4527-9c5a-cc39ae607708 deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:34:53.630619 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Delete of backup '0ee17d6a-58ae-4bdd-9100-172dbd9af093' for volume 'a0daa52e-7dba-4527-9c5a-cc39ae607708' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:34:53.636102 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:34:53.724642 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Created reservations ['e289a740-2084-4d75-adba-31ce9c45459a', 'e6b16a27-252b-4def-9d16-1bfad359e56f'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:34:53.948386 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-b7a11d7f-01d7-4d96-92f5-ae12c5889e97 tempest-VolumesBackupsAdminTest-354663248 None] Delete backup finished, backup 0ee17d6a-58ae-4bdd-9100-172dbd9af093 deleted.
Feb 12 21:34:58.299467 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Create backup started, backup: 8622758b-1127-4f0b-a744-f68cf8a82b4a volume: e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.
Feb 12 21:34:58.382704 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Call Volume Manager to get_backup_device for Backup(availability_zone=None,container=None,created_at=2022-02-12T21:34:57Z,data_timestamp=2022-02-12T21:34:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-2075372544',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=8622758b-1127-4f0b-a744-f68cf8a82b4a,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='4d6c242b7ff346f5b5cb6b75c4321618',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='c5e0e7a6b76b4cfeb2c032cfff336d9e',volume_id=e67ac3c6-5802-4c5c-88c9-0ee335dfea0a)
Feb 12 21:34:58.421739 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.31', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:58.423190 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.431244 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:58.431570 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[0791aed7-ff60-4e6d-b481-3c2f220c3bb0]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:d455a8a4a371\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:58.433476 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.440655 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8fc1f36c-3b01-4d21-ac8b-171adb54dd4d]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:34:58.441228 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:34:58.442358 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fc1f36c-3b01-4d21-ac8b-171adb54dd4d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:58.442836 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.449393 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[edb27062-718e-4e45-b291-9e9f65ff1e5e]: [Errno 2] No such file or directory: 'systool' {{(pid=161241) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]: Traceback (most recent call last):
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     ret = func(*f_args, **f_kwargs)
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     return func(*args, **kwargs)
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     return putils.execute(on_execute=on_execute,
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     obj = subprocess.Popen(cmd,
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 12 21:34:58.450627 ncdv-master-1 cinder-backup[134603]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Feb 12 21:34:58.451652 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[edb27062-718e-4e45-b291-9e9f65ff1e5e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:58.452425 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.462665 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:58.464811 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c17eb3f-55a5-4640-8f4e-653f74ed5629]: (4, ('/dev/sda1\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:58.464811 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.473692 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:58.474024 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cc6ed6d-97b0-45ce-8ada-2b803ffbe5a6]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:58.475222 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.482802 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=161241) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:58.483401 ncdv-master-1 cinder-backup[134603]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2a101d5-915d-4ac9-9eb3-46363e7ac42c]: (4, ('1c1633ff-5b55-c64b-8f90-b637d4724bdc\n', '')) {{(pid=161241) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Feb 12 21:34:58.484598 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Running cmd (subprocess): nvme version {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Feb 12 21:34:58.505138 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_concurrency.processutils [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=134603) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Feb 12 21:34:58.506443 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.utils [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] <== get_connector_properties: return (84ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.31', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:d455a8a4a371', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '1c1633ff-5b55-c64b-8f90-b637d4724bdc', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:42390e04-3ed2-4eda-9ba4-750df1486e3f'} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:59.204256 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connector [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Factory for rbd on None {{(pid=134603) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Feb 12 21:34:59.205801 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3900c85e50>, 'connection_properties': {'name': 'volumes/volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e67ac3c6-5802-4c5c-88c9-0ee335dfea0a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:34:59.206992 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.linuxrbd [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] opening connection to ceph cluster (timeout=-1). {{(pid=134603) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Feb 12 21:34:59.417131 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] <== connect_volume: return (211ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c79dc0>} {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:34:59.426936 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:34:59.501848 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Starting backup of volume='e67ac3c6-5802-4c5c-88c9-0ee335dfea0a'. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Feb 12 21:34:59.502882 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Volume file is RBD: attempting optimized backup {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Feb 12 21:34:59.788326 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Creating base image 'volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' {{(pid=134603) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Feb 12 21:34:59.788806 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134603) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Feb 12 21:35:00.755553 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Using --from-snap 'None' for incremental backup of volume e67ac3c6-5802-4c5c-88c9-0ee335dfea0a. {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Feb 12 21:35:00.756133 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Creating backup snapshot='backup.8622758b-1127-4f0b-a744-f68cf8a82b4a.snap.1644701700.7555792' {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Feb 12 21:35:02.972114 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Performing differential transfer from 'volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a' to 'volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' {{(pid=134603) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Feb 12 21:35:02.972764 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_kvree5lv --pool volumes volumes/volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a@backup.8622758b-1127-4f0b-a744-f68cf8a82b4a.snap.1644701700.7555792 -' into... {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Feb 12 21:35:02.972969 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' {{(pid=134603) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Feb 12 21:35:03.201000 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Base backup image='volume-8634339a-63d4-437c-aab7-c9cd3446b531.backup.91a00afe-73f8-4fb5-b868-f5cd70a56fc7' of volume 8634339a-63d4-437c-aab7-c9cd3446b531 deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:35:03.431270 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Delete of backup '91a00afe-73f8-4fb5-b868-f5cd70a56fc7' for volume '8634339a-63d4-437c-aab7-c9cd3446b531' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:35:03.507607 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Created reservations ['b92fa0f5-5406-4ca9-8c26-3bbc4a064954', '65c899ac-0b66-457d-be9b-f7ff8be26b19'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:35:03.645052 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-0827376e-d4ea-441b-a177-16a0fb84239d tempest-VolumesBackupsTest-835000994 None] Delete backup finished, backup 91a00afe-73f8-4fb5-b868-f5cd70a56fc7 deleted.
Feb 12 21:35:06.281799 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Differential backup transfer completed in 3.3097s {{(pid=134603) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Feb 12 21:35:06.283236 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Getting metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Feb 12 21:35:06.299618 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134603) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Feb 12 21:35:06.300289 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Getting metadata type 'volume-metadata' {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Feb 12 21:35:06.306957 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] No metadata type 'volume-metadata' available {{(pid=134603) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Feb 12 21:35:06.307341 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Getting metadata type 'volume-glance-metadata' {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Feb 12 21:35:06.313795 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.driver [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] No metadata type 'volume-glance-metadata' available {{(pid=134603) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Feb 12 21:35:06.320047 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Backing up metadata for volume e67ac3c6-5802-4c5c-88c9-0ee335dfea0a. {{(pid=134603) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Feb 12 21:35:06.491444 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Backup '8622758b-1127-4f0b-a744-f68cf8a82b4a' of volume e67ac3c6-5802-4c5c-88c9-0ee335dfea0a finished. {{(pid=134603) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Feb 12 21:35:06.491444 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f3900c85e50>, 'connection_properties': {'name': 'volumes/volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a', 'hosts': ['10.222.0.31'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e67ac3c6-5802-4c5c-88c9-0ee335dfea0a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f3900c79dc0>}, 'force': True, ': True}" {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Feb 12 21:35:07.129853 ncdv-master-1 cinder-backup[134603]: DEBUG os_brick.initiator.connectors.rbd [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] <== disconnect_volume: return (644ms) None {{(pid=134603) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Feb 12 21:35:07.691249 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-aef30e0b-add6-486f-b07b-7fa45c3574af tempest-VolumesBackupsAdminTest-354663248 None] Create backup finished. backup: 8622758b-1127-4f0b-a744-f68cf8a82b4a.
Feb 12 21:35:08.649820 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-6ba15f5d-06b6-48cb-844b-56fe08febf85 tempest-VolumesBackupsAdminTest-1031776151 None] Reset backup status started, backup_id: 8622758b-1127-4f0b-a744-f68cf8a82b4a, status: error.
Feb 12 21:35:08.650330 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-6ba15f5d-06b6-48cb-844b-56fe08febf85 tempest-VolumesBackupsAdminTest-1031776151 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver.
Feb 12 21:35:09.794023 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Delete backup started, backup: 8622758b-1127-4f0b-a744-f68cf8a82b4a.
Feb 12 21:35:09.799492 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Delete started for backup=8622758b-1127-4f0b-a744-f68cf8a82b4a {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:35:09.799492 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Trying diff format basename='volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' for backup base image of volume e67ac3c6-5802-4c5c-88c9-0ee335dfea0a. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:35:10.079806 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Looking for snapshot of backup base 'volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:35:10.080197 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Found snapshot 'backup.8622758b-1127-4f0b-a744-f68cf8a82b4a.snap.1644701700.7555792' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:35:10.080571 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Deleting backup snapshot='backup.8622758b-1127-4f0b-a744-f68cf8a82b4a.snap.1644701700.7555792' {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Feb 12 21:35:12.233147 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Deleting backup base image='volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' of volume e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.
Feb 12 21:35:13.722385 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:35:16.817262 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Base backup image='volume-e67ac3c6-5802-4c5c-88c9-0ee335dfea0a.backup.8622758b-1127-4f0b-a744-f68cf8a82b4a' of volume e67ac3c6-5802-4c5c-88c9-0ee335dfea0a deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:35:16.989946 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Delete of backup '8622758b-1127-4f0b-a744-f68cf8a82b4a' for volume 'e67ac3c6-5802-4c5c-88c9-0ee335dfea0a' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:35:17.097222 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Created reservations ['5daaa49b-7268-42c2-a591-753d62a81912', '8ed1f824-ede0-44a5-bb28-c6281ef2d1e2'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:35:17.278054 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-ecf63c7d-1cc2-4247-aee6-f15e34e8525a tempest-VolumesBackupsAdminTest-354663248 None] Delete backup finished, backup 8622758b-1127-4f0b-a744-f68cf8a82b4a deleted.
Feb 12 21:35:28.387091 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:35:28.387695 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:36:28.392305 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:36:28.392850 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:37:15.095971 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Delete backup started, backup: 278e59a9-78b0-48d1-ab85-d144c12c2de5.
Feb 12 21:37:15.097453 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Delete started for backup=278e59a9-78b0-48d1-ab85-d144c12c2de5 {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Feb 12 21:37:15.098324 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Trying diff format basename='volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' for backup base image of volume b3511a55-12b4-476c-b22f-f927dc31e59a. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Feb 12 21:37:15.317948 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Looking for snapshot of backup base 'volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Feb 12 21:37:15.318329 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Found snapshot 'backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808' {{(pid=134603) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Feb 12 21:37:15.318698 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Deleting backup snapshot='backup.278e59a9-78b0-48d1-ab85-d144c12c2de5.snap.1644701537.5630808' {{(pid=134603) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Feb 12 21:37:18.251704 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Deleting backup base image='volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' of volume b3511a55-12b4-476c-b22f-f927dc31e59a.
Feb 12 21:37:23.726841 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_db.sqlalchemy.engines [None req-23e1c87d-3413-441e-b43c-bd848196cfbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Feb 12 21:37:28.397779 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:37:28.398461 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:37:30.551149 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Base backup image='volume-b3511a55-12b4-476c-b22f-f927dc31e59a.backup.278e59a9-78b0-48d1-ab85-d144c12c2de5' of volume b3511a55-12b4-476c-b22f-f927dc31e59a deleted. {{(pid=134603) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Feb 12 21:37:30.736438 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.backup.drivers.ceph [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Delete of backup '278e59a9-78b0-48d1-ab85-d144c12c2de5' for volume 'b3511a55-12b4-476c-b22f-f927dc31e59a' finished. {{(pid=134603) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Feb 12 21:37:30.859021 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.quota [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Created reservations ['8e7d7395-c350-4b65-96a1-b59a6ed8274e', '1dc79d66-f00a-4f3e-b355-e5c7911a2247'] {{(pid=134603) reserve /opt/stack/cinder/cinder/quota.py:755}}
Feb 12 21:37:31.022068 ncdv-master-1 cinder-backup[134603]: INFO cinder.backup.manager [None req-7603163c-955d-4ab2-80af-6072a59ce664 tempest-TestVolumeBackupRestore-1020972158 None] Delete backup finished, backup 278e59a9-78b0-48d1-ab85-d144c12c2de5 deleted.
Feb 12 21:38:28.394247 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:38:28.395077 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:39:28.390896 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:39:28.391831 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:40:28.395803 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:40:28.396420 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:41:28.392985 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:41:28.394050 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:42:28.389882 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:42:28.390522 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:43:28.395386 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:43:28.396074 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:44:28.402015 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:44:28.402622 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:45:28.407419 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:45:28.408025 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:46:28.413529 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:46:28.414255 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:47:28.410054 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:47:28.410750 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:48:28.415323 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:48:28.415972 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:49:28.420586 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:49:28.421234 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:50:28.425375 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:50:28.426078 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:51:28.430508 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:51:28.431214 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:52:28.435911 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:52:28.436642 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:53:28.434086 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:53:28.434773 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:54:28.439176 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:54:28.439862 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:55:28.444453 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:55:28.445152 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:56:28.448769 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:56:28.449440 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:57:28.454587 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:57:28.455198 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:58:28.457670 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:58:28.458349 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 21:59:28.463003 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 21:59:28.463711 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:00:28.468468 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:00:28.469111 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:01:28.465657 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:01:28.466355 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:02:28.472321 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:02:28.473017 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:03:28.477508 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:03:28.478220 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:04:28.482395 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:04:28.483496 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:05:28.487926 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:05:28.488606 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:06:28.485618 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:06:28.486317 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:07:28.483453 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:07:28.484165 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:08:28.488953 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:08:28.489686 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:09:28.494357 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:09:28.494991 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:10:28.490937 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:10:28.491553 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:11:28.489881 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:11:28.491998 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:12:28.494958 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:12:28.495635 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:13:28.500257 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:13:28.500844 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:14:28.505236 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:14:28.505948 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:15:28.510314 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:15:28.510958 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:16:28.515016 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:16:28.515656 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:17:28.511633 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:17:28.512279 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:18:28.517537 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:18:28.518201 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:19:28.515827 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:19:28.516484 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:20:28.521013 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:20:28.521681 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Feb 12 22:21:28.525622 ncdv-master-1 cinder-backup[134603]: DEBUG oslo_service.periodic_task [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134603) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Feb 12 22:21:28.526872 ncdv-master-1 cinder-backup[134603]: DEBUG cinder.manager [None req-58049023-1475-4280-a10d-71eb7dc43384 None None] Notifying Schedulers of capabilities ... {{(pid=134603) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}