-- Logs begin at Sat 2022-01-22 20:01:15 UTC, end at Sat 2022-01-22 22:50:14 UTC. --
Jan 22 20:50:18.411433 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service.
Jan 22 20:50:21.518359 ncdv-master-1 cinder-backup[129420]: INFO cinder.cmd.backup [-] Backup running with 1 processes.
Jan 22 20:50:21.519708 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=129420) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}}
Jan 22 20:50:21.520246 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=129420) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}}
Jan 22 20:50:21.685637 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_db.sqlalchemy.engines [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129420) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 20:50:21.897232 ncdv-master-1 cinder-backup[129420]: INFO cinder.rpc [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version.
Jan 22 20:50:21.901113 ncdv-master-1 cinder-backup[129420]: INFO cinder.rpc [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version.
Jan 22 20:50:21.904956 ncdv-master-1 cinder-backup[129420]: INFO cinder.rpc [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version.
Jan 22 20:50:21.908756 ncdv-master-1 cinder-backup[129420]: INFO cinder.rpc [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version.
Jan 22 20:50:21.913132 ncdv-master-1 cinder-backup[129420]: INFO cinder.rpc [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
Jan 22 20:50:21.917059 ncdv-master-1 cinder-backup[129420]: INFO cinder.rpc [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version.
Jan 22 20:50:21.963989 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_db.sqlalchemy.engines [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129420) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 20:50:22.121146 ncdv-master-1 cinder-backup[129420]: INFO oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Starting 1 workers
Jan 22 20:50:22.128114 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Started child 129865 {{(pid=129420) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Jan 22 20:50:22.129178 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Full set of CONF: {{(pid=129420) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}}
Jan 22 20:50:22.129677 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] ******************************************************************************** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}}
Jan 22 20:50:22.130121 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] Configuration options gathered from: {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}}
Jan 22 20:50:22.130486 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}}
Jan 22 20:50:22.131227 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}}
Jan 22 20:50:22.131620 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] ================================================================================ {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}}
Jan 22 20:50:22.132496 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] allow_availability_zone_fallback = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.133044 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] allow_compression_on_image_upload = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.133954 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] allowed_direct_url_schemes     = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.134503 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] api_rate_limit                 = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.135002 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] auth_strategy                  = keystone {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.135405 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_availability_zone      = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.137780 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_api_class               = cinder.backup.api.API {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.138215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_chunk_size         = 134217728 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.138649 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_conf               = /etc/ceph/ceph.conf {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.139029 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_image_journals     = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.139417 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_pool               = backups {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.142235 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_stripe_count       = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.142691 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_stripe_unit        = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.144658 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_ceph_user               = cinder-bak {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.149319 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_driver                  = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.150519 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_driver_init_check_interval = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.152748 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_driver_stats_polling_interval = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.153152 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_manager                 = cinder.backup.manager.BackupManager {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.153596 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_max_operations          = 15 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.158131 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_metadata_version        = 2 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.158761 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_name_template           = backup-%s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.159127 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_native_threads_pool_size = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.159897 ncdv-master-1 cinder-backup[129865]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0)
Jan 22 20:50:22.160668 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_object_number_per_notification = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.161611 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_service_inithost_offload = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.162043 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_timer_interval          = 120 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.162444 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_use_same_host           = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.164297 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_use_temp_snapshot       = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.164834 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backup_workers                 = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.165239 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] chap_password                  = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.165655 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] chap_username                  =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.168328 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.169608 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_internal_tenant_project_id = 98c62088157141ffa8ace45b5c19d03f {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.170001 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_internal_tenant_user_id = adcb6a8499884b4896281bcbd183086e {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.171416 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] compression_format             = gzip {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.172894 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] config_dir                     = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.173652 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] config_file                    = ['/etc/cinder/cinder.conf'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.174071 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] config_source                  = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.175605 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] consistencygroup_api_class     = cinder.consistencygroup.api.API {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.176997 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] control_exchange               = openstack {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.177800 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] debug                          = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.178535 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] default_availability_zone      = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.180584 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] default_group_type             = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.181273 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.181880 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] default_volume_type            = ceph {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.182269 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] driver_client_cert             = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.183999 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] driver_client_cert_key         = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.184364 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] driver_data_namespace          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.185403 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] driver_ssl_cert_path           = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.185797 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] driver_ssl_cert_verify         = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.189360 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] driver_use_ssl                 = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.189636 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] enable_new_services            = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.190007 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] enable_unsupported_driver      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.190398 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] enabled_backends               = ['ceph'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.191990 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] enforce_multipath_for_image_xfer = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.192349 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] filter_function                = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.192711 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_api_insecure            = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.193090 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_api_servers             = ['http://10.222.0.100/image'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.196366 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_api_ssl_compression     = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.196737 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_ca_certificates_file    = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.197683 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_catalog_info            = image:glance:publicURL {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.199338 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_certfile                = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.200301 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.201846 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_keyfile                 = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.202229 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_num_retries             = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.202580 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] glance_request_timeout         = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.202950 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] goodness_function              = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.203322 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] graceful_shutdown_timeout      = 5 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.204277 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] group_api_class                = cinder.group.api.API {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.204647 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] host                           = ncdv-master-1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.209118 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iet_conf                       = /etc/iet/ietd.conf {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.210076 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_compress_on_upload       = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.210612 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_conversion_address_space_limit = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.213106 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_conversion_cpu_limit     = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.213693 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_conversion_dir           = /opt/stack/data/cinder/conversion {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.214109 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_upload_use_cinder_backend = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.214433 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_upload_use_internal_tenant = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.214788 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_volume_cache_enabled     = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.215141 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_volume_cache_max_count   = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.217372 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] image_volume_cache_max_size_gb = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.217771 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] instance_format                = [instance: %(uuid)s]  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.219284 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.219913 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iscsi_iotype                   = fileio {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.220301 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iscsi_secondary_ip_addresses   = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.220673 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iscsi_target_flags             =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.223417 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.lockutils [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] Lock "cleanup_incomplete_backups_129420" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.001s {{(pid=129865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}}
Jan 22 20:50:22.223766 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] Cleaning up incomplete backup operations.
Jan 22 20:50:22.224351 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iscsi_write_cache              = on {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.224739 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iser_helper                    = tgtadm {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.225462 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iser_ip_address                = 10.222.0.100 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.226144 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iser_port                      = 3260 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.228386 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] iser_target_prefix             = iqn.2010-10.org.openstack: {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.228785 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_config_append              = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.229414 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.229943 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_dir                        = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.230299 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_file                       = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.230592 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_options                    = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.230941 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_rotate_interval            = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.232581 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_rotate_interval_type       = days {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.232969 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] log_rotation_type              = none {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.236195 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.236514 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.237017 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.237400 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.237714 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.238111 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] max_age                        = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.240776 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] max_logfile_count              = 30 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.244192 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 20:50:22.244490 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] max_logfile_size_mb            = 200 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.244944 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] max_over_subscription_ratio    = 20.0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.247104 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] message_reap_interval          = 86400 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.247487 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] message_ttl                    = 2592000 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.247881 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] monkey_patch                   = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.248261 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] monkey_patch_modules           = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.251824 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] my_ip                          = 10.222.0.100 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.252195 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] no_snapshot_gb_quota           = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.252554 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] num_iser_scan_tries            = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.252934 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] num_shell_tries                = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.253303 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] num_volume_device_scan_tries   = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.256317 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] nvmet_ns_id                    = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.256823 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] nvmet_port_id                  = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.257341 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_max_limit                = 1000 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.257738 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_volume_ext_list          = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.258166 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.258558 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_volume_listen            = 0.0.0.0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.260491 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_volume_listen_port       = 8776 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.260858 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_volume_use_ssl           = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.261230 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] osapi_volume_workers           = 2 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.263691 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] per_volume_size_limit          = -1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.264070 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] periodic_fuzzy_delay           = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.266375 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] periodic_interval              = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.266887 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] public_endpoint                = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.267548 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] publish_errors                 = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.267940 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_backup_gigabytes         = 1000 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.268338 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_backups                  = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.268725 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_consistencygroups        = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.269910 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_driver                   = cinder.quota.DbQuotaDriver {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.272109 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_gigabytes                = 1000 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.272507 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_groups                   = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.272906 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_snapshots                = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.273295 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] quota_volumes                  = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.273681 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] rate_limit_burst               = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.274052 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] rate_limit_except_level        = CRITICAL {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.274403 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] rate_limit_interval            = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.274765 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] replication_device             = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.275128 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] report_discard_supported       = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.275491 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] report_interval                = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.275926 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] reservation_clean_interval     = 86400 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.276233 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] reservation_expire             = 86400 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.280906 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] reserved_percentage            = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.282283 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] resource_query_filters_file    = /etc/cinder/resource_filters.json {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.282675 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] restore_discard_excess_bytes   = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.283052 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] rootwrap_config                = /etc/cinder/rootwrap.conf {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.283452 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] rpc_response_timeout           = 120 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.283818 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] run_external_periodic_tasks    = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.287263 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] scheduler_manager              = cinder.scheduler.manager.SchedulerManager {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.288009 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] scst_target_driver             = iscsi {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.288381 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] scst_target_iqn_name           = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.288686 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_down_time              = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.289102 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] snapshot_name_template         = snapshot-%s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.289457 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] split_loggers                  = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.289782 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] state_path                     = /opt/stack/data/cinder {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.290144 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] storage_availability_zone      = nova {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.294860 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] storage_protocol               = iscsi {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.297603 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] syslog_log_facility            = LOG_USER {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.297603 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] target_helper                  = lioadm {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.300178 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] target_ip_address              = 10.222.0.100 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.300623 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] target_port                    = 3260 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.301008 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] target_prefix                  = iqn.2010-10.org.openstack: {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.301464 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] target_protocol                = iscsi {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.307181 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] trace_flags                    = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.307595 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] transfer_api_class             = cinder.transfer.api.API {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.308259 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] transport_url                  = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.308629 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] until_refresh                  = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.308995 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_chap_auth                  = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.309411 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_default_quota_class        = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.309805 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_eventlog                   = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.310164 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_journal                    = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.310630 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_json                       = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.311003 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_multipath_for_image_xfer   = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.311355 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_stderr                     = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.311697 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] use_syslog                     = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.312087 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] verify_glance_signatures       = enabled {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.312446 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_api_class               = cinder.volume.api.API {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.312782 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_backend_name            = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.317374 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_clear                   = zero {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.317778 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_clear_ionice            = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.318429 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_clear_size              = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.318798 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_copy_blkio_cgroup_name  = cinder-volume-copy {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.319152 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_copy_bps_limit          = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.319516 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_dd_blocksize            = 1M {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.319883 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_manager                 = cinder.volume.manager.VolumeManager {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.320242 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_name_template           = volume-%s {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.320581 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volume_usage_audit_period      = month {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.325456 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] volumes_dir                    = /opt/stack/data/cinder/volumes {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.325813 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] watch_log_file                 = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 22 20:50:22.326335 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.326740 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.327269 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.enforce_new_defaults = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.327649 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.enforce_scope      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.329390 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.policy_default_rule = default {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.329819 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.330202 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.policy_file        = policy.yaml {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.330592 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.331107 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.331398 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.331756 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.332105 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.332478 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_concurrency.disable_process_locking = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.332852 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_concurrency.lock_path     = /opt/stack/data/cinder {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.333280 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.admin_password = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.333675 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.334032 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.admin_token = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.334372 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.admin_user  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.334742 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_admin_prefix =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.335091 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_host   = 127.0.0.1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.335451 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_port   = 35357 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.335828 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_protocol = https {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.336273 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_section = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.336639 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_type   = password {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.337033 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_uri    = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.337444 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.auth_version = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.338059 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.cache       = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.338736 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.cafile      = /opt/stack/data/ca-bundle.pem {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.339039 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.certfile    = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.339387 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.delay_auth_decision = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.339767 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.340133 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.http_connect_timeout = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.340431 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.340786 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.identity_uri = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.341229 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.include_service_catalog = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.341676 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.insecure    = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.342031 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.interface   = public {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.342432 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.keyfile     = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.342793 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.343135 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.343477 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.343828 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.344684 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.345026 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.345405 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.345823 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.346231 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.346567 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.region_name = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.347374 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.347730 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.service_token_roles_required = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.348067 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.service_type = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.348512 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.token_cache_time = 300 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.349002 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.349450 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.backend                  = dogpile.cache.null {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.349927 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.backend_argument         = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.350293 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.config_prefix            = cache.oslo {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.350661 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.debug_cache_backend      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.351006 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.enabled                  = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.351454 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.expiration_time          = 600 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.351820 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_dead_retry      = 300 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_pool_maxsize    = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.proxies                  = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360861 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.tls_allowed_ciphers      = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360861 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.tls_cafile               = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360861 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.tls_certfile             = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360861 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.tls_enabled              = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360861 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cache.tls_keyfile              = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.360861 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] coordination.backend_url       = etcd3+http://10.222.0.100:2379 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.backend               = sqlalchemy {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.connection            = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.connection_debug      = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.connection_parameters =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.connection_recycle_time = 3600 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.connection_trace      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361215 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.db_inc_retry_interval = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361652 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.db_max_retries        = 20 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361652 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.db_max_retry_interval = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361652 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.db_retry_interval     = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.361829 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.max_overflow          = 50 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.362857 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.max_pool_size         = 5 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.362857 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.max_retries           = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.362857 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.mysql_enable_ndb      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.363081 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.363374 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.pool_timeout          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.363726 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.retry_interval        = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.364200 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.slave_connection      = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.364637 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.sqlite_synchronous    = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.365220 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] database.use_db_reconnect      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.365719 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.369547 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.369547 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.369547 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.369547 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.369547 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.connection_string     = messaging:// {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.369547 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.enabled               = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.370019 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.es_doc_type           = notification {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.370019 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.es_scroll_size        = 10000 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.370019 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.es_scroll_time        = 2m {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.370019 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.filter_error_trace    = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.371028 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.371781 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.sentinel_service_name = mymaster {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.372279 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.socket_timeout        = 0.1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.372758 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] profiler.trace_sqlalchemy      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.373326 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] privsep_osbrick.capabilities   = [21] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.373862 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] privsep_osbrick.group          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.374350 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] privsep_osbrick.helper_command = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.374883 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] privsep_osbrick.logger_name    = oslo_privsep.daemon {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.375379 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.375887 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] privsep_osbrick.user           = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.376426 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_sys_admin.capabilities  = [0, 1, 2, 3, 12, 21] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.376918 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_sys_admin.group         = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.377411 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_sys_admin.helper_command = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.378075 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_sys_admin.logger_name   = oslo_privsep.daemon {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.378688 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.379321 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] cinder_sys_admin.user          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.380031 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.auth_section      = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.380537 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.auth_type         = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.381028 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.cafile            = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.381644 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.certfile          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.382182 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.collect_timing    = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.382681 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.insecure          = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.383203 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.keyfile           = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.383744 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.send_service_user_token = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.384266 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.split_loggers     = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.384772 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] service_user.timeout           = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.385347 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.backend_availability_zone = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.385897 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.chap_password = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.386452 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.chap_username =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.386964 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.387470 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.388100 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.388636 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.driver_client_cert = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.389287 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.driver_client_cert_key = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.389941 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.driver_data_namespace = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.390495 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.391040 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.391786 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.driver_use_ssl = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.392307 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.enable_deferred_deletion = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.392828 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.enable_unsupported_driver = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.393399 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.394140 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.filter_function = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.394655 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.goodness_function = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.395194 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iet_conf      = /etc/iet/ietd.conf {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.395708 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.396221 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.396740 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.397254 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.397804 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.399156 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iscsi_iotype  = fileio {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.399882 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.400536 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iscsi_target_flags =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.401788 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iscsi_write_cache = on {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.402630 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iser_helper   = tgtadm {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.404154 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iser_ip_address = 10.222.0.100 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.404719 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iser_port     = 3260 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.405414 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.406108 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.407241 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.407919 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.num_shell_tries = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.408766 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.409438 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.nvmet_ns_id   = 10 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.410181 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.nvmet_port_id = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.410771 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.411337 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rados_connection_interval = 5 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.411996 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rados_connection_retries = 3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.412576 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_ceph_conf =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.413135 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.413807 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.414637 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.415171 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.415757 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_pool      = rbd {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.416316 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_secret_uuid = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.416878 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.417427 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.rbd_user      = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.418049 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.418605 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.replication_device = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.419342 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.report_discard_supported = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.419896 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.420556 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.reserved_percentage = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.421113 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.scst_target_driver = iscsi {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.421741 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.scst_target_iqn_name = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.423311 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.storage_protocol = iscsi {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.424008 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.target_helper = tgtadm {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.424710 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.target_ip_address = 10.222.0.100 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.425274 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.target_port   = 3260 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.425893 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.426479 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.target_protocol = iscsi {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.427023 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.trace_flags   = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.427579 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.unique_fqdn_network = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.428139 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.use_chap_auth = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.428707 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.429441 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_backend_name = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.430169 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_clear  = zero {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.430839 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_clear_ionice = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.431402 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_clear_size = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.432107 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.432772 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.433336 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.434023 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] backend_defaults.volumes_dir   = /opt/stack/data/cinder/volumes {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.434603 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_reports.file_event_handler = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.435192 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.435757 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_reports.log_dir           = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.436368 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.436973 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.438364 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.439827 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.440404 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.441090 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.441777 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.442366 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.443023 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.443567 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.444176 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.444742 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.445325 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.445900 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.446448 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.447239 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.447803 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.448368 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.448916 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.449811 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.450474 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.451389 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.ssl      = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.451962 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.452533 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.453196 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.454598 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.455232 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_notifications.driver = [] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.455808 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_notifications.retry = -1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.456400 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.456984 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] oslo_messaging_notifications.transport_url = **** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.457665 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] key_manager.backend            = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.458296 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.458844 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.barbican_api_version  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.459385 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.barbican_endpoint     = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.460205 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.barbican_endpoint_type = public {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.460779 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.barbican_region_name  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.461472 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.cafile                = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.462058 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.certfile              = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.462634 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.collect_timing        = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.463364 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.insecure              = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.463926 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.keyfile               = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.464672 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.number_of_retries     = 60 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.465224 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.retry_delay           = 1 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.465795 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.send_service_user_token = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.466561 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.split_loggers         = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.468418 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.timeout               = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.468987 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.verify_ssl            = True {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.470326 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican.verify_ssl_path       = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.470917 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.auth_section = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.471633 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.auth_type = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.472192 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.cafile   = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.472814 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.certfile = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.473873 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.collect_timing = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.474444 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.insecure = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.474996 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.keyfile  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.475684 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.split_loggers = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.476086 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] barbican_service_user.timeout  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.476552 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.approle_role_id          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.476963 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.approle_secret_id        = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.477365 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.cafile                   = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.477793 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.certfile                 = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.478208 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.collect_timing           = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.478644 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.insecure                 = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.479125 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.keyfile                  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.479552 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.kv_mountpoint            = secret {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.479909 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.kv_version               = 2 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.480378 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.namespace                = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.480715 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.root_token_id            = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.481232 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.split_loggers            = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.481718 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.ssl_ca_crt_file          = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.488204 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.timeout                  = None {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.488204 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.use_ssl                  = False {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.488204 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 22 20:50:22.488204 ncdv-master-1 cinder-backup[129420]: DEBUG oslo_service.service [None req-2ca50e14-989b-481a-a266-04106b6f25e7 None None] ******************************************************************************** {{(pid=129420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}}
Jan 22 20:50:22.530007 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.lockutils [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] Lock "cleanup_incomplete_backups_129420" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.307s {{(pid=129865) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}}
Jan 22 20:50:22.537603 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 20:50:22.544639 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:50:22.572787 ncdv-master-1 cinder-backup[129865]: INFO cinder.keymgr.migration [None req-89fa020b-61dd-46a5-948f-62ee37889ec9 None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Jan 22 20:50:22.604844 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.service [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] Creating RPC server for service cinder-backup {{(pid=129865) start /opt/stack/cinder/cinder/service.py:231}}
Jan 22 20:50:22.619576 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.service [None req-0118bd9c-97a8-402a-8cf6-4f81be731c04 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=129865) start /opt/stack/cinder/cinder/service.py:237}}
Jan 22 20:51:56.651041 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:51:56.651728 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:52:56.656077 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:52:56.656750 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:53:56.651838 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:53:56.652523 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:54:56.657065 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:54:56.657778 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:55:56.652930 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:55:56.653611 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:56:56.657475 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:56:56.658172 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:57:56.661928 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:57:56.662579 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:58:56.667002 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:58:56.667750 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 20:59:56.678389 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 20:59:56.678389 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:00:56.677940 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:00:56.678529 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:01:56.675559 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:01:56.676196 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:02:56.671464 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:02:56.673319 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:03:56.668761 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:03:56.670071 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:04:56.664711 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:04:56.665455 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:05:56.663840 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:05:56.664943 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:06:56.665272 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:06:56.666939 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:07:56.664304 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:07:56.665134 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:08:56.661364 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:08:56.662022 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:09:56.658218 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:09:56.660486 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:10:56.663604 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:10:56.664210 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:11:56.667710 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:11:56.668979 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:12:56.665675 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:12:56.667689 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:13:56.665287 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:13:56.665950 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:14:56.670151 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:14:56.675324 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:15:56.677224 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:15:56.677962 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:16:56.682396 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:16:56.683179 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:17:56.686200 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:17:56.687239 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:18:56.691666 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:18:56.697916 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:19:56.690221 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:19:56.690938 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:21:56.697090 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:21:56.698151 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:22:56.698270 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:22:56.701034 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:23:56.696388 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:23:56.696388 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:24:56.694247 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:24:56.694963 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:25:56.700163 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:25:56.703168 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:26:54.986851 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Create backup started, backup: ee45e9ce-68cc-42ca-aec8-986c59aa62a0 volume: e9b2332f-81f1-4153-b8b4-cb705451aa3a.
Jan 22 21:26:55.091536 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:26:55.102283 ncdv-master-1 cinder-backup[129865]: INFO oslo.privsep.daemon [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp5picrn3j/privsep.sock']
Jan 22 21:26:55.199980 ncdv-master-1 sudo[163859]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmp5picrn3j/privsep.sock
Jan 22 21:26:55.202171 ncdv-master-1 sudo[163859]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jan 22 21:26:56.700343 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:26:56.701839 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:27:02.101194 ncdv-master-1 sudo[163859]: pam_unix(sudo:session): session closed for user root
Jan 22 21:27:02.106736 ncdv-master-1 cinder-backup[129865]: INFO oslo.privsep.daemon [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Spawned new privsep daemon via rootwrap
Jan 22 21:27:02.107351 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Accepted privsep connection to /tmp/tmp5picrn3j/privsep.sock {{(pid=129865) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}}
Jan 22 21:27:02.109594 ncdv-master-1 cinder-backup[129865]: INFO oslo.privsep.daemon [-] privsep daemon starting
Jan 22 21:27:02.109873 ncdv-master-1 cinder-backup[129865]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Jan 22 21:27:02.110141 ncdv-master-1 cinder-backup[129865]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Jan 22 21:27:02.110403 ncdv-master-1 cinder-backup[129865]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 164079
Jan 22 21:27:02.118455 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[71deceba-fe37-4389-a257-f7ff39ddb1fb]: (2,) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:02.869105 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:27:02.893575 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:02.925903 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.035s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:27:02.926300 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[60228ba0-ac79-4239-bc57-d97a9d3ea084]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:02.928170 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:02.961859 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[55155012-1691-4887-844f-b5de5add7f50]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:27:03.004666 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[55155012-1691-4887-844f-b5de5add7f50]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:03.011465 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:03.024959 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f5104182-97d5-4331-b9c1-e8578bd8998e]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:27:03.025987 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:27:03.027326 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5104182-97d5-4331-b9c1-e8578bd8998e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:03.029277 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:03.073828 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.044s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:27:03.074359 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[346c02ad-525c-4c29-a4e5-331f63d9f31f]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:03.075814 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:03.170750 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.089s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:27:03.170750 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfca79ac-8fb3-49be-82e0-78f62accf436]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:03.170750 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:03.191162 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.023s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:27:03.193918 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7b27d19-cfe1-46a6-804f-17c3bfd38efd]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:27:03.195658 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:27:03.243921 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] CMD "nvme version" returned: 0 in 0.048s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:27:03.260217 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] <== get_connector_properties: return (8167ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:27:04.631138 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:27:04.635484 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd4787e20>, 'connection_properties': {'name': 'volumes/volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e9b2332f-81f1-4153-b8b4-cb705451aa3a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:27:04.637782 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:27:04.985332 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] <== connect_volume: return (348ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd47188e0>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:27:05.117413 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Starting backup of volume='e9b2332f-81f1-4153-b8b4-cb705451aa3a'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:27:05.119075 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Volume file is RBD: attempting optimized backup {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 22 21:27:05.217113 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Creating base image 'volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' {{(pid=129865) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 22 21:27:05.218130 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:27:09.139926 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Using --from-snap 'None' for incremental backup of volume e9b2332f-81f1-4153-b8b4-cb705451aa3a. {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 22 21:27:09.139926 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Creating backup snapshot='backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0.snap.1642886829.1307814' {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 22 21:27:12.425833 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Performing differential transfer from 'volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a' to 'volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:27:12.438376 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_3ba7a71q --pool volumes volumes/volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a@backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0.snap.1642886829.1307814 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:27:12.438376 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:27:16.174848 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Differential backup transfer completed in 3.7483s {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 22 21:27:16.182453 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:27:16.204449 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:27:16.204872 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:27:16.230840 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:27:16.235562 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:27:16.254149 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] No metadata type 'volume-glance-metadata' available {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 22 21:27:16.254149 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Backing up metadata for volume e9b2332f-81f1-4153-b8b4-cb705451aa3a. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:27:16.525281 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Backup 'ee45e9ce-68cc-42ca-aec8-986c59aa62a0' of volume e9b2332f-81f1-4153-b8b4-cb705451aa3a finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:27:16.528926 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd4787e20>, 'connection_properties': {'name': 'volumes/volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e9b2332f-81f1-4153-b8b4-cb705451aa3a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd47188e0>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:27:17.534453 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] <== disconnect_volume: return (1005ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:27:17.863517 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-ab5d9243-cd7f-4652-8c89-ab9fd7ce4b5a tempest-VolumesBackupsV39Test-1645624774 None] Create backup finished. backup: ee45e9ce-68cc-42ca-aec8-986c59aa62a0.
Jan 22 21:27:19.323526 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Delete backup started, backup: ee45e9ce-68cc-42ca-aec8-986c59aa62a0.
Jan 22 21:27:19.324823 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Delete started for backup=ee45e9ce-68cc-42ca-aec8-986c59aa62a0 {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:27:19.325320 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Trying diff format basename='volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' for backup base image of volume e9b2332f-81f1-4153-b8b4-cb705451aa3a. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:27:19.683634 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Looking for snapshot of backup base 'volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:27:19.684283 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Found snapshot 'backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0.snap.1642886829.1307814' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:27:19.684557 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Deleting backup snapshot='backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0.snap.1642886829.1307814' {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 22 21:27:22.867612 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:27:22.984404 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Deleting backup base image='volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' of volume e9b2332f-81f1-4153-b8b4-cb705451aa3a.
Jan 22 21:27:30.870412 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Base backup image='volume-e9b2332f-81f1-4153-b8b4-cb705451aa3a.backup.ee45e9ce-68cc-42ca-aec8-986c59aa62a0' of volume e9b2332f-81f1-4153-b8b4-cb705451aa3a deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:27:31.229007 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Delete of backup 'ee45e9ce-68cc-42ca-aec8-986c59aa62a0' for volume 'e9b2332f-81f1-4153-b8b4-cb705451aa3a' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:27:31.462315 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Created reservations ['dcd5a8cc-15f1-44ab-a76d-b6f0bd7d0f1a', '652c2a26-a3ed-4c24-a6a6-914eb370f3a4'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:27:31.772721 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-260cf793-2f2d-45e0-bca7-cf9aac732dbb tempest-VolumesBackupsV39Test-1645624774 None] Delete backup finished, backup ee45e9ce-68cc-42ca-aec8-986c59aa62a0 deleted.
Jan 22 21:27:56.697995 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:27:56.698666 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:28:56.694233 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:28:56.698743 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:29:45.679421 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Create backup started, backup: 3917be5c-da21-4a67-8617-ddf7be83d8a5 volume: c78173a9-3798-4b6d-91d8-061c98e61c28.
Jan 22 21:29:45.771347 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:29:45.778565 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:45.803427 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.030s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:29:45.803724 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[5214bed6-7834-4520-bc33-16b29e476d9a]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:29:45.807120 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:45.816554 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e625a6b9-c9fc-44fc-9ae8-aae089e24f7f]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:29:45.816853 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:29:45.817732 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[e625a6b9-c9fc-44fc-9ae8-aae089e24f7f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:29:45.822312 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:45.840367 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[617923f0-8d8a-4707-9ebd-9732e1a231ea]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:29:45.844450 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:29:45.845854 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[617923f0-8d8a-4707-9ebd-9732e1a231ea]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:29:45.848715 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:45.882894 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.034s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:29:45.883703 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c6aa1ca-b2ac-4e82-a29b-cf554cfb06fe]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:29:45.888675 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:45.923248 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.036s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:29:45.923769 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa02d669-d34e-4639-a690-a40741550484]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:29:45.927655 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:45.947571 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.022s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:29:45.948153 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[87ffa3d8-e8e7-4059-935a-e974fbe338b9]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:29:45.949244 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:29:46.018411 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] CMD "nvme version" returned: 0 in 0.069s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:29:46.019801 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] <== get_connector_properties: return (247ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:29:47.059057 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:29:47.060535 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd477c760>, 'connection_properties': {'name': 'volumes/volume-c78173a9-3798-4b6d-91d8-061c98e61c28', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'c78173a9-3798-4b6d-91d8-061c98e61c28', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:29:47.062563 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:29:47.430909 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] <== connect_volume: return (369ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd46e3970>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:29:47.440119 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:29:47.539214 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Starting backup of volume='c78173a9-3798-4b6d-91d8-061c98e61c28'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:29:47.541828 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Volume file is RBD: attempting optimized backup {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 22 21:29:47.584472 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Creating base image 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' {{(pid=129865) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 22 21:29:47.584882 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:29:48.418198 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Using --from-snap 'None' for incremental backup of volume c78173a9-3798-4b6d-91d8-061c98e61c28. {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 22 21:29:48.418745 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Creating backup snapshot='backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255' {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 22 21:29:51.315419 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Performing differential transfer from 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28' to 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:29:51.316014 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_agapboqi --pool volumes volumes/volume-c78173a9-3798-4b6d-91d8-061c98e61c28@backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:29:51.316188 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:29:55.235133 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Differential backup transfer completed in 3.9191s {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 22 21:29:55.235812 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:29:55.258602 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:29:55.259014 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:29:55.267812 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:29:55.268432 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:29:55.279223 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] No metadata type 'volume-glance-metadata' available {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 22 21:29:55.281377 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Backing up metadata for volume c78173a9-3798-4b6d-91d8-061c98e61c28. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:29:55.533258 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Backup '3917be5c-da21-4a67-8617-ddf7be83d8a5' of volume c78173a9-3798-4b6d-91d8-061c98e61c28 finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:29:55.535056 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd477c760>, 'connection_properties': {'name': 'volumes/volume-c78173a9-3798-4b6d-91d8-061c98e61c28', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'c78173a9-3798-4b6d-91d8-061c98e61c28', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd46e3970>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:29:55.906257 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] <== disconnect_volume: return (367ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:29:56.126891 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-aabf0d23-9080-4be8-957c-ccdc51108fcc tempest-VolumesBackupsAdminTest-1905346629 None] Create backup finished. backup: 3917be5c-da21-4a67-8617-ddf7be83d8a5.
Jan 22 21:29:56.695117 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:29:56.695660 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:29:57.516075 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-cb9a7be8-f086-4ec2-a35d-51e6f5d3d5e8 tempest-VolumesBackupsAdminTest-686646659 None] Export record started, backup: 3917be5c-da21-4a67-8617-ddf7be83d8a5.
Jan 22 21:29:57.524818 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-cb9a7be8-f086-4ec2-a35d-51e6f5d3d5e8 tempest-VolumesBackupsAdminTest-686646659 None] Export record finished, backup 3917be5c-da21-4a67-8617-ddf7be83d8a5 exported.
Jan 22 21:29:58.297707 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-050816a9-77ef-4e56-b51b-bc11f92ab3d5 tempest-VolumesBackupsAdminTest-686646659 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiMDRkZWJjNzYtNDkwNS00NDA2LThhNGEtYmU5ZjhiNjYzNTlhIiwgInVzZXJfaWQiOiAiMjcwM2M1NTBmOTIzNDFiNWFiNzJhZWZjNTQwMmY3ODYiLCAicHJvamVjdF9pZCI6ICJjN2RlMjhjMjA2OGU0ODc1OTU2NjMwODE2OGY3YzE0NSIsICJ2b2x1bWVfaWQiOiAiYzc4MTczYTktMzc5OC00YjZkLTkxZDgtMDYxYzk4ZTYxYzI4IiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC0xMjc5MTU3MTQyIiwgImRpc3BsYXlfZGVzY3JpcHRpb24iOiBudWxsLCAic2VydmljZV9tZXRhZGF0YSI6ICJ7XCJiYXNlXCI6IFwidm9sdW1lLWM3ODE3M2E5LTM3OTgtNGI2ZC05MWQ4LTA2MWM5OGU2MWMyOC5iYWNrdXAuMzkxN2JlNWMtZGEyMS00YTY3LTg2MTctZGRmN2JlODNkOGE1XCJ9IiwgInNlcnZpY2UiOiAiY2luZGVyLmJhY2t1cC5kcml2ZXJzLmNlcGguQ2VwaEJhY2t1cERyaXZlciIsICJvYmplY3RfY291bnQiOiAwLCAidGVtcF92b2x1bWVfaWQiOiBudWxsLCAidGVtcF9zbmFwc2hvdF9pZCI6IG51bGwsICJudW1fZGVwZW5kZW50X2JhY2t1cHMiOiAwLCAic25hcHNob3RfaWQiOiBudWxsLCAiZGF0YV90aW1lc3RhbXAiOiAiMjAyMi0wMS0yMlQyMToyOTo0NVoiLCAicmVzdG9yZV92b2x1bWVfaWQiOiBudWxsLCAibWV0YWRhdGEiOiB7fSwgImVuY3J5cHRpb25fa2V5X2lkIjogbnVsbCwgImNyZWF0ZWRfYXQiOiAiMjAyMi0wMS0yMlQyMToyOTo0NVoiLCAidXBkYXRlZF9hdCI6ICIyMDIyLTAxLTIyVDIxOjI5OjU2WiIsICJkZWxldGVkX2F0IjogbnVsbCwgImRlbGV0ZWQiOiBmYWxzZX0=.
Jan 22 21:29:58.306652 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-050816a9-77ef-4e56-b51b-bc11f92ab3d5 tempest-VolumesBackupsAdminTest-686646659 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:29:58.550144 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-050816a9-77ef-4e56-b51b-bc11f92ab3d5 tempest-VolumesBackupsAdminTest-686646659 None] Import record id 04debc76-4905-4406-8a4a-be9f8b66359a metadata from driver finished.
Jan 22 21:30:02.676349 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Restore backup started, backup: 3917be5c-da21-4a67-8617-ddf7be83d8a5 volume: 500e9832-3e26-4318-aa5b-87fa84482aa7.
Jan 22 21:30:02.689957 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:30:02.694206 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.704242 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:02.704638 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[a364762e-bc7c-494d-bf8e-88a7131d57e1]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:02.705975 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.711980 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6cfb4c7c-9473-46ae-b5dc-026579bcb7dc]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:30:02.712713 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:30:02.713786 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cfb4c7c-9473-46ae-b5dc-026579bcb7dc]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:02.714916 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.722087 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2ca6e0ae-d16f-4845-aac9-e1da47690906]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:30:02.722580 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:30:02.723543 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ca6e0ae-d16f-4845-aac9-e1da47690906]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:02.724308 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.733573 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:02.734028 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[37b6e581-488b-4482-9743-808f0358e894]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:02.735205 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.744445 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:02.744890 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4e60131-8a0e-40df-bac2-74fd38c6f54b]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:02.746066 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.753873 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:02.754202 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[67f49f84-600f-4256-b84a-219cf178c2e1]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:02.754979 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:02.774337 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:02.775479 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] <== get_connector_properties: return (85ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:30:02.880275 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:30:03.540871 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:30:03.547247 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd4779cd0>, 'connection_properties': {'name': 'volumes/volume-500e9832-3e26-4318-aa5b-87fa84482aa7', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '500e9832-3e26-4318-aa5b-87fa84482aa7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:30:03.548480 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:30:03.953648 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] <== connect_volume: return (405ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd461fdc0>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:30:03.967456 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Starting restore from Ceph backup=3917be5c-da21-4a67-8617-ddf7be83d8a5 to volume=volume-500e9832-3e26-4318-aa5b-87fa84482aa7 {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Jan 22 21:30:04.097253 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Looking for snapshot of backup base 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:30:04.097901 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Found snapshot 'backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:30:04.109790 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Volume file is RBD. {{(pid=129865) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Jan 22 21:30:04.187342 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Attempting differential restore. {{(pid=129865) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Jan 22 21:30:04.188141 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Attempting incremental restore from base='volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' snap='backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255' {{(pid=129865) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Jan 22 21:30:04.189039 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Performing differential transfer from 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' to 'volume-500e9832-3e26-4318-aa5b-87fa84482aa7' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:30:04.189624 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5@backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:30:04.190148 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd__50jf_85 --pool volumes - volumes/volume-500e9832-3e26-4318-aa5b-87fa84482aa7' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:30:06.692662 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Restore transfer completed in 2.5034s {{(pid=129865) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Jan 22 21:30:06.695221 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Jan 22 21:30:06.974625 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Restoring volume base metadata {{(pid=129865) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Jan 22 21:30:07.161748 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] No metadata of type 'volume-metadata' to restore {{(pid=129865) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 22 21:30:07.166030 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=129865) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 22 21:30:07.166943 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Restore to volume 500e9832-3e26-4318-aa5b-87fa84482aa7 finished successfully. {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Jan 22 21:30:07.168840 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd4779cd0>, 'connection_properties': {'name': 'volumes/volume-500e9832-3e26-4318-aa5b-87fa84482aa7', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '500e9832-3e26-4318-aa5b-87fa84482aa7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd461fdc0>}, 'force': True, ': False}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:30:07.352903 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] <== disconnect_volume: return (182ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:30:09.990522 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.manager [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Encryption key ID for volume 500e9832-3e26-4318-aa5b-87fa84482aa7 already matches encryption key ID in backup 3917be5c-da21-4a67-8617-ddf7be83d8a5. {{(pid=129865) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Jan 22 21:30:10.238625 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-d4eb3dad-e04e-44f6-9e83-0a164867dbae tempest-VolumesBackupsAdminTest-1905346629 None] Finished restoring backup 3917be5c-da21-4a67-8617-ddf7be83d8a5 to volume 500e9832-3e26-4318-aa5b-87fa84482aa7.
Jan 22 21:30:11.481627 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Delete backup started, backup: 04debc76-4905-4406-8a4a-be9f8b66359a.
Jan 22 21:30:11.484193 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Delete started for backup=04debc76-4905-4406-8a4a-be9f8b66359a {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:30:11.484501 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Trying diff format basename='volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' for backup base image of volume c78173a9-3798-4b6d-91d8-061c98e61c28. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:30:11.735438 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Looking for snapshot of backup base 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:30:11.735438 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Backup '04debc76-4905-4406-8a4a-be9f8b66359a' has no snapshot {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}}
Jan 22 21:30:11.735438 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] No backup snapshot to delete {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Jan 22 21:30:11.837252 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Backup base image of volume c78173a9-3798-4b6d-91d8-061c98e61c28 still has 1 snapshots so skipping base image delete.
Jan 22 21:30:11.955539 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Metadata backup object 'backup.04debc76-4905-4406-8a4a-be9f8b66359a.meta' not found - ignoring {{(pid=129865) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}}
Jan 22 21:30:11.959779 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Delete of backup '04debc76-4905-4406-8a4a-be9f8b66359a' for volume 'c78173a9-3798-4b6d-91d8-061c98e61c28' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:30:11.967868 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:30:12.075880 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Created reservations ['f530aa8f-2d8c-4b72-a7a3-2c2893280739', 'd0a7d73d-4682-41c1-8498-306b237b6bee'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:30:12.272546 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-dcb6d941-dedb-494a-a4f6-4c93a3b204cf tempest-VolumesBackupsAdminTest-686646659 None] Delete backup finished, backup 04debc76-4905-4406-8a4a-be9f8b66359a deleted.
Jan 22 21:30:12.662811 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Delete backup started, backup: 3917be5c-da21-4a67-8617-ddf7be83d8a5.
Jan 22 21:30:12.664625 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Delete started for backup=3917be5c-da21-4a67-8617-ddf7be83d8a5 {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:30:12.665030 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Trying diff format basename='volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' for backup base image of volume c78173a9-3798-4b6d-91d8-061c98e61c28. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:30:12.925931 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Looking for snapshot of backup base 'volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:30:12.926393 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Found snapshot 'backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:30:12.926823 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Deleting backup snapshot='backup.3917be5c-da21-4a67-8617-ddf7be83d8a5.snap.1642886988.4182255' {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 22 21:30:12.930561 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:30:15.165366 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Deleting backup base image='volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' of volume c78173a9-3798-4b6d-91d8-061c98e61c28.
Jan 22 21:30:22.866210 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Base backup image='volume-c78173a9-3798-4b6d-91d8-061c98e61c28.backup.3917be5c-da21-4a67-8617-ddf7be83d8a5' of volume c78173a9-3798-4b6d-91d8-061c98e61c28 deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:30:23.644474 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Delete of backup '3917be5c-da21-4a67-8617-ddf7be83d8a5' for volume 'c78173a9-3798-4b6d-91d8-061c98e61c28' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:30:23.744531 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Created reservations ['f7a0f57c-10d0-439e-a4a7-751f09fb57a5', 'e46dd60a-4ae3-4caa-894f-3f867fb12a3c'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:30:23.964725 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-78b9e6d6-9c34-430d-8ce0-244e04e04669 tempest-VolumesBackupsAdminTest-1905346629 None] Delete backup finished, backup 3917be5c-da21-4a67-8617-ddf7be83d8a5 deleted.
Jan 22 21:30:31.347571 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Create backup started, backup: d978f760-c10e-43ac-bfe7-63b5bc57371f volume: 9543bc04-28d3-48b7-9540-82e1613a4a5c.
Jan 22 21:30:31.461878 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:30:31.465545 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.491653 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.026s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:31.492203 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[70e15de0-deb4-4b0c-9bbc-b35729c78c61]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:31.502918 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.512878 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4810cdd8-a444-458d-a1af-6bd4beb1f636]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:30:31.513182 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:30:31.516225 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[4810cdd8-a444-458d-a1af-6bd4beb1f636]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:31.518096 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.547394 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[521be16b-34ac-4321-83a6-109b83252a08]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:30:31.547860 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:30:31.548853 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[521be16b-34ac-4321-83a6-109b83252a08]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:31.558841 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.582095 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:31.587098 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[53cc4537-0045-4dab-ad0a-f607e95bb179]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:31.587098 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.611390 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.026s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:31.611920 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d33c7e5-9c09-4c7f-a0c5-8590adb5cb8c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:31.613892 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.644803 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.031s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:31.645101 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[47a412ed-3698-421b-81c7-bdeb8c1d8346]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:30:31.645900 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:30:31.701649 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] CMD "nvme version" returned: 0 in 0.055s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:30:31.705250 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] <== get_connector_properties: return (241ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:30:32.930645 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:30:33.047916 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:30:33.049600 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd47349d0>, 'connection_properties': {'name': 'volumes/volume-9543bc04-28d3-48b7-9540-82e1613a4a5c', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '9543bc04-28d3-48b7-9540-82e1613a4a5c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:30:33.050796 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:30:33.444709 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] <== connect_volume: return (394ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd45b0e20>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:30:33.565252 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Starting backup of volume='9543bc04-28d3-48b7-9540-82e1613a4a5c'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:30:33.566909 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Volume file is RBD: attempting optimized backup {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 22 21:30:33.620180 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Creating base image 'volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' {{(pid=129865) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 22 21:30:33.620653 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:30:34.580334 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Using --from-snap 'None' for incremental backup of volume 9543bc04-28d3-48b7-9540-82e1613a4a5c. {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 22 21:30:34.581018 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Creating backup snapshot='backup.d978f760-c10e-43ac-bfe7-63b5bc57371f.snap.1642887034.5804853' {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 22 21:30:36.598832 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Performing differential transfer from 'volume-9543bc04-28d3-48b7-9540-82e1613a4a5c' to 'volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:30:36.599481 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_i8fk6e97 --pool volumes volumes/volume-9543bc04-28d3-48b7-9540-82e1613a4a5c@backup.d978f760-c10e-43ac-bfe7-63b5bc57371f.snap.1642887034.5804853 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:30:36.599617 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:30:39.831326 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Differential backup transfer completed in 3.2321s {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 22 21:30:39.832477 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:30:39.848140 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:30:39.857969 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:30:39.857969 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:30:39.857969 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:30:39.864643 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] No metadata type 'volume-glance-metadata' available {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 22 21:30:39.865213 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Backing up metadata for volume 9543bc04-28d3-48b7-9540-82e1613a4a5c. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:30:40.134057 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Backup 'd978f760-c10e-43ac-bfe7-63b5bc57371f' of volume 9543bc04-28d3-48b7-9540-82e1613a4a5c finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:30:40.136153 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd47349d0>, 'connection_properties': {'name': 'volumes/volume-9543bc04-28d3-48b7-9540-82e1613a4a5c', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '9543bc04-28d3-48b7-9540-82e1613a4a5c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd45b0e20>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:30:40.868602 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] <== disconnect_volume: return (731ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:30:41.141695 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-33761fd4-d4ca-4018-a914-9d67212c09db tempest-VolumesBackupsAdminTest-1905346629 None] Create backup finished. backup: d978f760-c10e-43ac-bfe7-63b5bc57371f.
Jan 22 21:30:41.779294 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-a1d9d952-a906-4ede-ae38-bd777aea25cb tempest-VolumesBackupsAdminTest-686646659 None] Reset backup status started, backup_id: d978f760-c10e-43ac-bfe7-63b5bc57371f, status: error.
Jan 22 21:30:41.779686 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-a1d9d952-a906-4ede-ae38-bd777aea25cb tempest-VolumesBackupsAdminTest-686646659 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver.
Jan 22 21:30:43.015919 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Delete backup started, backup: d978f760-c10e-43ac-bfe7-63b5bc57371f.
Jan 22 21:30:43.023705 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Delete started for backup=d978f760-c10e-43ac-bfe7-63b5bc57371f {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:30:43.024377 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Trying diff format basename='volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' for backup base image of volume 9543bc04-28d3-48b7-9540-82e1613a4a5c. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:30:43.260520 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Looking for snapshot of backup base 'volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:30:43.261164 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Found snapshot 'backup.d978f760-c10e-43ac-bfe7-63b5bc57371f.snap.1642887034.5804853' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:30:43.261722 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Deleting backup snapshot='backup.d978f760-c10e-43ac-bfe7-63b5bc57371f.snap.1642887034.5804853' {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 22 21:30:45.126303 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Deleting backup base image='volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' of volume 9543bc04-28d3-48b7-9540-82e1613a4a5c.
Jan 22 21:30:49.680473 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Base backup image='volume-9543bc04-28d3-48b7-9540-82e1613a4a5c.backup.d978f760-c10e-43ac-bfe7-63b5bc57371f' of volume 9543bc04-28d3-48b7-9540-82e1613a4a5c deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:30:49.933644 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Delete of backup 'd978f760-c10e-43ac-bfe7-63b5bc57371f' for volume '9543bc04-28d3-48b7-9540-82e1613a4a5c' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:30:49.940113 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:30:50.033835 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Created reservations ['946f1364-61ce-4576-930d-e3dd939e46d1', '3a74b407-fb75-4290-b990-dc472ce42074'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:30:50.227399 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-96319d33-09f8-4f4d-b636-a80e2cd6d26d tempest-VolumesBackupsAdminTest-1905346629 None] Delete backup finished, backup d978f760-c10e-43ac-bfe7-63b5bc57371f deleted.
Jan 22 21:30:56.691760 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:30:56.692386 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:31:00.546299 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Create backup started, backup: c19a6022-8ffb-4052-8a65-fd47eb636ae0 volume: 5c51668d-fc20-4f01-a879-3813357f7a73.
Jan 22 21:31:00.701894 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:31:00.717772 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.732094 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:00.732593 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6590179-b346-4219-a712-a9e2e9785344]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:00.734324 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.744084 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[295ae2fa-3b9b-4fee-b406-1081680c42f6]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[295ae2fa-3b9b-4fee-b406-1081680c42f6]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:00.750951 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.760157 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[44944e1f-3a4e-4d0c-a72b-58946cb1907f]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:31:00.760719 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:31:00.762260 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[44944e1f-3a4e-4d0c-a72b-58946cb1907f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:00.764220 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.774981 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.010s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:00.775306 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[02449dee-79c9-4c39-aa36-98fc9b88be14]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:00.779271 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.786573 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:00.786896 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[6657ef5b-8a46-4737-8546-73cc5a33ef05]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:00.788315 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.798007 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:00.798723 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a36c763-415a-4bf6-b01e-7d51e6308ed5]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:00.799635 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:00.822128 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] CMD "nvme version" returned: 0 in 0.022s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:00.823321 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] <== get_connector_properties: return (114ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:31:02.924103 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:31:04.233384 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:31:04.238125 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd48a2580>, 'connection_properties': {'name': 'volumes/volume-5c51668d-fc20-4f01-a879-3813357f7a73', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '5c51668d-fc20-4f01-a879-3813357f7a73', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:31:04.239702 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:31:04.502139 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] <== connect_volume: return (262ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd473db50>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:31:04.569194 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Starting backup of volume='5c51668d-fc20-4f01-a879-3813357f7a73'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:31:04.571314 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Volume file is RBD: attempting optimized backup {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 22 21:31:04.595644 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Creating base image 'volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' {{(pid=129865) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 22 21:31:04.596229 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:31:05.522350 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Using --from-snap 'None' for incremental backup of volume 5c51668d-fc20-4f01-a879-3813357f7a73. {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 22 21:31:05.522834 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Creating backup snapshot='backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0.snap.1642887065.5223463' {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 22 21:31:07.608345 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Performing differential transfer from 'volume-5c51668d-fc20-4f01-a879-3813357f7a73' to 'volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:31:07.608953 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_26zkp4yl --pool volumes volumes/volume-5c51668d-fc20-4f01-a879-3813357f7a73@backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0.snap.1642887065.5223463 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:31:07.610345 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:31:11.898963 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Differential backup transfer completed in 4.2901s {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 22 21:31:11.899584 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:31:11.930354 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:31:11.931959 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:31:11.946627 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:31:11.947205 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:31:11.956524 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] No metadata type 'volume-glance-metadata' available {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 22 21:31:11.957589 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Backing up metadata for volume 5c51668d-fc20-4f01-a879-3813357f7a73. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:31:12.404759 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Backup 'c19a6022-8ffb-4052-8a65-fd47eb636ae0' of volume 5c51668d-fc20-4f01-a879-3813357f7a73 finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:31:12.407015 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd48a2580>, 'connection_properties': {'name': 'volumes/volume-5c51668d-fc20-4f01-a879-3813357f7a73', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '5c51668d-fc20-4f01-a879-3813357f7a73', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd473db50>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:31:12.644531 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] <== disconnect_volume: return (235ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:31:12.890295 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-c0323dde-c64c-4188-a24b-f1e011ebe944 tempest-VolumesBackupsTest-1274070825 None] Create backup finished. backup: c19a6022-8ffb-4052-8a65-fd47eb636ae0.
Jan 22 21:31:14.076455 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Delete backup started, backup: c19a6022-8ffb-4052-8a65-fd47eb636ae0.
Jan 22 21:31:14.087679 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Delete started for backup=c19a6022-8ffb-4052-8a65-fd47eb636ae0 {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:31:14.088121 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Trying diff format basename='volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' for backup base image of volume 5c51668d-fc20-4f01-a879-3813357f7a73. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:31:14.311388 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Looking for snapshot of backup base 'volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:31:14.312215 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Found snapshot 'backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0.snap.1642887065.5223463' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:31:14.312852 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Deleting backup snapshot='backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0.snap.1642887065.5223463' {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 22 21:31:16.269198 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Deleting backup base image='volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' of volume 5c51668d-fc20-4f01-a879-3813357f7a73.
Jan 22 21:31:20.623066 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Base backup image='volume-5c51668d-fc20-4f01-a879-3813357f7a73.backup.c19a6022-8ffb-4052-8a65-fd47eb636ae0' of volume 5c51668d-fc20-4f01-a879-3813357f7a73 deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:31:20.742859 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Delete of backup 'c19a6022-8ffb-4052-8a65-fd47eb636ae0' for volume '5c51668d-fc20-4f01-a879-3813357f7a73' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:31:20.749708 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:31:20.844451 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Created reservations ['c5579829-8e7f-4b85-8a3e-61d85b372dce', '260815a4-3f1a-48ed-a8f3-0a633bf6fab2'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:31:21.052150 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-0b5c9fa7-42ad-4639-acbf-cdce6a30f675 tempest-VolumesBackupsTest-1274070825 None] Delete backup finished, backup c19a6022-8ffb-4052-8a65-fd47eb636ae0 deleted.
Jan 22 21:31:39.074322 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Create backup started, backup: 2f5b12fd-27aa-456a-9138-a7d22a5a9d91 volume: 8ce51a79-e21d-40d7-9de1-28e5ee775236.
Jan 22 21:31:39.148676 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:31:39.150180 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.157930 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:39.159045 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad2ec063-8b74-4600-af61-af67e6b6cc90]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:39.160519 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.166192 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[517f143e-71f1-4a41-a560-2bbc0e40fd19]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:31:39.166866 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:31:39.168017 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[517f143e-71f1-4a41-a560-2bbc0e40fd19]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:39.170333 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.174729 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0242f454-7cbf-4bdc-9636-99e4858d16fc]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:31:39.175344 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:31:39.176507 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[0242f454-7cbf-4bdc-9636-99e4858d16fc]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:39.177099 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.185249 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:39.185820 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d066a0c-04fc-405d-8b91-275f7988c602]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:39.187077 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.195513 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:39.195856 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2e14c49-93c4-4ac7-8f3d-2527e0cf7e70]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:39.196961 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.203820 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:39.204207 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf4ae2c6-729d-45ab-97cd-256030bff11c]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:31:39.204970 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:31:39.223850 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:31:39.224861 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] <== get_connector_properties: return (75ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:31:40.063517 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:31:40.065830 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd48cba00>, 'connection_properties': {'name': 'volumes/volume-8ce51a79-e21d-40d7-9de1-28e5ee775236', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8ce51a79-e21d-40d7-9de1-28e5ee775236', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:31:40.066886 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:31:40.255893 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] <== connect_volume: return (189ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd3d4b280>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:31:40.268698 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:31:40.364736 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Starting backup of volume='8ce51a79-e21d-40d7-9de1-28e5ee775236'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:31:40.365753 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Volume file is RBD: attempting optimized backup {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 22 21:31:40.407614 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Creating base image 'volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' {{(pid=129865) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 22 21:31:40.408043 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:31:40.834017 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Using --from-snap 'None' for incremental backup of volume 8ce51a79-e21d-40d7-9de1-28e5ee775236. {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 22 21:31:40.834431 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Creating backup snapshot='backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414' {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 22 21:31:41.910871 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Performing differential transfer from 'volume-8ce51a79-e21d-40d7-9de1-28e5ee775236' to 'volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:31:41.911523 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_q_6u08ud --pool volumes volumes/volume-8ce51a79-e21d-40d7-9de1-28e5ee775236@backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:31:41.911905 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:31:56.693455 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:31:56.694218 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:32:17.494816 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Create backup started, backup: ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0 volume: f6238c78-261a-421a-b7a2-43273813620f snapshot: 7c63570e-c956-483e-9263-a774676c22d8.
Jan 22 21:32:17.555509 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:32:17.556913 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.564369 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:17.564697 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1b1a95c-f07d-4dd6-8e30-f227a0d03a9e]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:17.566048 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.571859 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0d40c085-512c-496f-8f0c-789b84c4d32c]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:32:17.572529 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:32:17.573768 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d40c085-512c-496f-8f0c-789b84c4d32c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:17.574207 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.581273 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4c1e0fb3-b4a5-42b7-8106-3bbab7f69415]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:32:17.582030 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:32:17.584978 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c1e0fb3-b4a5-42b7-8106-3bbab7f69415]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:17.585915 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.596754 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.010s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:17.597243 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[d46bb790-fa28-482b-b181-a02c121b06b1]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:17.598532 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.609630 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.011s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:17.610021 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[839011f4-5c94-4e8c-8816-d565db876d0c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:17.611274 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.619468 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:17.621025 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a080828-e055-47a3-9e38-45334af6c9d2]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:17.626680 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:17.647239 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:17.648421 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] <== get_connector_properties: return (92ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:32:21.637087 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:32:21.638861 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd3d7ed60>, 'connection_properties': {'name': 'volumes/volume-6d37bad7-ecfe-4e0a-a976-2471b21418e9', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6d37bad7-ecfe-4e0a-a976-2471b21418e9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:32:21.640511 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:32:21.956838 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] <== connect_volume: return (313ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd3d81d90>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:32:21.964906 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:32:22.044466 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Starting backup of volume='f6238c78-261a-421a-b7a2-43273813620f'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:32:22.069264 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:32:22.069811 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Creating backup base image='volume-f6238c78-261a-421a-b7a2-43273813620f.backup.base' for volume f6238c78-261a-421a-b7a2-43273813620f. {{(pid=129865) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}}
Jan 22 21:32:23.787881 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Copying data from volume f6238c78-261a-421a-b7a2-43273813620f. {{(pid=129865) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}}
Jan 22 21:32:24.136650 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferring data between 'volume-f6238c78-261a-421a-b7a2-43273813620f' and 'volume-f6238c78-261a-421a-b7a2-43273813620f.backup.base' {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}}
Jan 22 21:32:24.137041 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] 8 chunks of 134217728 bytes to be transferred {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}}
Jan 22 21:32:31.440727 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Differential backup transfer completed in 49.5300s {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 22 21:32:31.443952 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:32:31.452300 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:32:31.452707 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:32:31.458876 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:32:31.459257 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:32:31.467030 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Jan 22 21:32:31.467644 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Backing up metadata for volume 8ce51a79-e21d-40d7-9de1-28e5ee775236. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:32:31.956490 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Backup '2f5b12fd-27aa-456a-9138-a7d22a5a9d91' of volume 8ce51a79-e21d-40d7-9de1-28e5ee775236 finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:32:31.958091 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd48cba00>, 'connection_properties': {'name': 'volumes/volume-8ce51a79-e21d-40d7-9de1-28e5ee775236', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8ce51a79-e21d-40d7-9de1-28e5ee775236', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd3d4b280>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:32:33.010236 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] <== disconnect_volume: return (1050ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:32:33.558983 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-3e219e5f-9d24-4df4-a147-fdc234cd5f9c tempest-VolumesBackupsTest-1274070825 None] Create backup finished. backup: 2f5b12fd-27aa-456a-9138-a7d22a5a9d91.
Jan 22 21:32:36.170889 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 1 of 8 (10892K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:32:37.227254 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Restore backup started, backup: 2f5b12fd-27aa-456a-9138-a7d22a5a9d91 volume: ac9572a9-a61b-4baa-bd8e-fe21c648011a.
Jan 22 21:32:37.241238 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:32:37.243751 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.252037 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:37.252374 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[c84e320a-4b8c-4958-8a7d-8bed5eba1070]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:37.253725 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.260263 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0fac76e6-06e3-4918-afcd-5423bc73c1ab]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:32:37.261194 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:32:37.266338 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fac76e6-06e3-4918-afcd-5423bc73c1ab]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:37.266338 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.268482 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4c1f64cb-a53e-43d4-b475-e9a013efb481]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:32:37.269069 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:32:37.270318 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c1f64cb-a53e-43d4-b475-e9a013efb481]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:37.270844 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.279353 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:37.279680 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[a48c9639-4052-460a-9193-c0edbdb1af85]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:37.280840 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.291024 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:37.291351 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f3fab26-02e6-4f2e-87bb-3b7ce129f5f3]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:37.292527 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.299231 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:37.299551 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[093d1326-6085-41f6-b579-e8ec71dcfdda]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:32:37.300326 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:32:37.334652 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] CMD "nvme version" returned: 0 in 0.034s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:32:37.335624 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] <== get_connector_properties: return (93ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:32:38.040462 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:32:38.042052 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd48a09d0>, 'connection_properties': {'name': 'volumes/volume-ac9572a9-a61b-4baa-bd8e-fe21c648011a', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ac9572a9-a61b-4baa-bd8e-fe21c648011a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:32:38.043083 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:32:38.393437 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] <== connect_volume: return (350ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd3cde190>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:32:38.403307 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:32:38.415195 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Starting restore from Ceph backup=2f5b12fd-27aa-456a-9138-a7d22a5a9d91 to volume=volume-ac9572a9-a61b-4baa-bd8e-fe21c648011a {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Jan 22 21:32:38.563146 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Looking for snapshot of backup base 'volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:32:38.563528 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Found snapshot 'backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:32:38.567989 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Volume file is RBD. {{(pid=129865) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Jan 22 21:32:41.291460 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Attempting differential restore. {{(pid=129865) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Jan 22 21:32:41.292160 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Attempting incremental restore from base='volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' snap='backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414' {{(pid=129865) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Jan 22 21:32:41.292318 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Performing differential transfer from 'volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' to 'volume-ac9572a9-a61b-4baa-bd8e-fe21c648011a' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:32:41.292717 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91@backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:32:41.293106 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_wse3lizm --pool volumes - volumes/volume-ac9572a9-a61b-4baa-bd8e-fe21c648011a' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:32:48.255652 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 2 of 8 (10846K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:32:56.689354 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:32:56.690073 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:33:07.789068 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 3 of 8 (6710K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:33:28.217481 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 4 of 8 (6416K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:33:49.738607 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 5 of 8 (6090K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:33:56.694388 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:33:56.695137 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:34:06.521283 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Restore transfer completed in 85.2287s {{(pid=129865) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Jan 22 21:34:06.522298 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Jan 22 21:34:06.548783 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Restoring volume base metadata {{(pid=129865) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Jan 22 21:34:06.622789 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] No metadata of type 'volume-metadata' to restore {{(pid=129865) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 22 21:34:06.623087 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Restoring volume glance metadata {{(pid=129865) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Jan 22 21:34:07.100908 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 6 of 8 (7549K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:34:07.877376 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Restore to volume ac9572a9-a61b-4baa-bd8e-fe21c648011a finished successfully. {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Jan 22 21:34:07.878997 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd48a09d0>, 'connection_properties': {'name': 'volumes/volume-ac9572a9-a61b-4baa-bd8e-fe21c648011a', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ac9572a9-a61b-4baa-bd8e-fe21c648011a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd3cde190>}, 'force': True, ': False}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:34:08.119589 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] <== disconnect_volume: return (239ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:34:08.175188 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.manager [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Encryption key ID for volume ac9572a9-a61b-4baa-bd8e-fe21c648011a already matches encryption key ID in backup 2f5b12fd-27aa-456a-9138-a7d22a5a9d91. {{(pid=129865) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Jan 22 21:34:08.466946 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-d04bb3e9-455d-414b-af61-94c87fe4b1ad tempest-VolumesBackupsTest-1274070825 None] Finished restoring backup 2f5b12fd-27aa-456a-9138-a7d22a5a9d91 to volume ac9572a9-a61b-4baa-bd8e-fe21c648011a.
Jan 22 21:34:18.705648 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 7 of 8 (11295K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:34:31.072341 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Transferred chunk 8 of 8 (10599K/s) {{(pid=129865) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 22 21:34:31.883202 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:34:31.895891 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:34:31.896390 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:34:31.903095 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:34:31.903483 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:34:31.910192 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] No metadata type 'volume-glance-metadata' available {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 22 21:34:31.910761 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Backing up metadata for volume f6238c78-261a-421a-b7a2-43273813620f. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:34:32.285357 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Backup 'ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0' of volume f6238c78-261a-421a-b7a2-43273813620f finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:34:32.287013 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd3d7ed60>, 'connection_properties': {'name': 'volumes/volume-6d37bad7-ecfe-4e0a-a976-2471b21418e9', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6d37bad7-ecfe-4e0a-a976-2471b21418e9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd3d81d90>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:34:32.422558 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] <== disconnect_volume: return (135ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:34:32.429479 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Delete backup started, backup: 2f5b12fd-27aa-456a-9138-a7d22a5a9d91.
Jan 22 21:34:32.431807 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Delete started for backup=2f5b12fd-27aa-456a-9138-a7d22a5a9d91 {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:34:32.432229 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Trying diff format basename='volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' for backup base image of volume 8ce51a79-e21d-40d7-9de1-28e5ee775236. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:34:32.499677 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:34:32.591805 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Looking for snapshot of backup base 'volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:34:32.592218 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Found snapshot 'backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:34:32.592593 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Deleting backup snapshot='backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91.snap.1642887100.8340414' {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 22 21:34:32.954147 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-b2de814d-2e8d-4c28-a9aa-830f188a0e41 tempest-VolumesSnapshotTestJSON-664893185 None] Create backup finished. backup: ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0.
Jan 22 21:34:34.113798 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Delete backup started, backup: ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0.
Jan 22 21:34:34.113798 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Delete started for backup=ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0 {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:34:34.114417 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Trying diff format basename='volume-f6238c78-261a-421a-b7a2-43273813620f.backup.ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0' for backup base image of volume f6238c78-261a-421a-b7a2-43273813620f. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:34:34.149700 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Image 'volume-f6238c78-261a-421a-b7a2-43273813620f.backup.ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0' not found - trying diff format name {{(pid=129865) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}}
Jan 22 21:34:34.353628 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Looking for snapshot of backup base 'volume-f6238c78-261a-421a-b7a2-43273813620f.backup.base' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:34:34.353946 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Backup base 'volume-f6238c78-261a-421a-b7a2-43273813620f.backup.base' has no snapshots {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}}
Jan 22 21:34:34.354316 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] No backup snapshot to delete {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Jan 22 21:34:34.543682 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Deleting backup base image='volume-f6238c78-261a-421a-b7a2-43273813620f.backup.base' of volume f6238c78-261a-421a-b7a2-43273813620f.
Jan 22 21:34:36.530612 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Deleting backup base image='volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' of volume 8ce51a79-e21d-40d7-9de1-28e5ee775236.
Jan 22 21:34:43.050288 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:34:47.889699 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Base backup image='volume-8ce51a79-e21d-40d7-9de1-28e5ee775236.backup.2f5b12fd-27aa-456a-9138-a7d22a5a9d91' of volume 8ce51a79-e21d-40d7-9de1-28e5ee775236 deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:34:48.105986 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Delete of backup '2f5b12fd-27aa-456a-9138-a7d22a5a9d91' for volume '8ce51a79-e21d-40d7-9de1-28e5ee775236' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:34:48.205721 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Created reservations ['6fe7b4d6-2740-42c6-bff4-ae0bc0f96445', 'bc438a23-a06e-4c48-b665-ad4ab541ef37'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:34:48.426440 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-c64eb500-6498-4a68-b43c-7bbf7a2e7082 tempest-VolumesBackupsTest-1274070825 None] Delete backup finished, backup 2f5b12fd-27aa-456a-9138-a7d22a5a9d91 deleted.
Jan 22 21:34:48.672885 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Base backup image='volume-f6238c78-261a-421a-b7a2-43273813620f.backup.base' of volume f6238c78-261a-421a-b7a2-43273813620f deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:34:48.807738 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Delete of backup 'ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0' for volume 'f6238c78-261a-421a-b7a2-43273813620f' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:34:48.991863 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Created reservations ['dc992556-587f-46ed-82b1-1c09714aac85', '221d1c6d-8c2f-4603-b9be-ce15048fa43f'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:34:49.237561 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-5f5a308f-209d-46dc-94af-3e8d54bbd07b tempest-VolumesSnapshotTestJSON-664893185 None] Delete backup finished, backup ae512b60-10ca-4c46-9fbe-d6d7c8f40fc0 deleted.
Jan 22 21:34:56.699105 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:34:56.699796 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:35:56.695625 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:35:56.696348 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:36:56.691751 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:36:56.692425 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:37:56.697047 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:37:56.697750 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:38:56.692585 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:38:56.693347 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:39:56.698691 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:39:56.699405 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:40:56.704194 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:40:56.704888 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:41:56.706355 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:41:56.707071 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:42:56.703152 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:42:56.706117 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:43:56.709566 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:43:56.710351 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:44:56.705562 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:44:56.706198 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:45:56.702603 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:45:56.703245 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:46:56.707481 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:46:56.708094 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:47:56.704775 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:47:56.705355 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:48:56.709458 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:48:56.710202 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:49:22.520955 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Create backup started, backup: 0974b291-264a-43d8-a350-04db1b3ede47 volume: 62f19c54-bca2-42c3-b76d-cd55e33e35d7.
Jan 22 21:49:22.602280 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:49:22.603705 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.612997 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:49:22.615256 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[21c7a7f6-1afb-48a3-8df4-ef026c5ff9cb]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:49:22.615256 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.621470 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[07d6991c-3b2d-4057-9c68-817c3f016c5b]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:49:22.622119 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:49:22.623352 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[07d6991c-3b2d-4057-9c68-817c3f016c5b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:49:22.623745 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.629488 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[fe5344bf-1521-40a8-9c42-0e3684f1daf7]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:49:22.630109 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:49:22.631341 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe5344bf-1521-40a8-9c42-0e3684f1daf7]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:49:22.631731 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.654953 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.023s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:49:22.655610 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6f17057-3372-490c-9d97-62a5f89e869a]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:49:22.656971 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.672299 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.015s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:49:22.672626 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[43b7b25d-d4b7-4e01-abbc-23535162a187]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:49:22.673831 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.680771 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:49:22.681101 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[500fc186-4527-44d2-ada2-71c86a20573e]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:49:22.681895 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:49:22.713587 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] CMD "nvme version" returned: 0 in 0.031s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:49:22.714686 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] <== get_connector_properties: return (111ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:49:23.156806 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:49:23.490492 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:49:23.491121 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd4718fd0>, 'connection_properties': {'name': 'volumes/volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '62f19c54-bca2-42c3-b76d-cd55e33e35d7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:49:23.492234 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:49:23.779551 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] <== connect_volume: return (287ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd4594a90>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:49:23.876821 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Starting backup of volume='62f19c54-bca2-42c3-b76d-cd55e33e35d7'. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 22 21:49:23.877905 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Volume file is RBD: attempting optimized backup {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 22 21:49:23.899487 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Creating base image 'volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' {{(pid=129865) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 22 21:49:23.899892 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129865) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 22 21:49:24.285988 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Using --from-snap 'None' for incremental backup of volume 62f19c54-bca2-42c3-b76d-cd55e33e35d7. {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 22 21:49:24.286489 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Creating backup snapshot='backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172' {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 22 21:49:25.402484 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Performing differential transfer from 'volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7' to 'volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:49:25.403052 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_18fg7ozr --pool volumes volumes/volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7@backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:49:25.403291 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:49:56.714830 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:49:56.715579 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:50:08.141749 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Differential backup transfer completed in 42.7391s {{(pid=129865) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 22 21:50:08.142472 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Getting metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 22 21:50:08.153618 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129865) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 22 21:50:08.154052 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Getting metadata type 'volume-metadata' {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 22 21:50:08.161030 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] No metadata type 'volume-metadata' available {{(pid=129865) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 22 21:50:08.161411 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Getting metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 22 21:50:08.168228 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=129865) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Jan 22 21:50:08.168800 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Backing up metadata for volume 62f19c54-bca2-42c3-b76d-cd55e33e35d7. {{(pid=129865) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 22 21:50:08.328603 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Backup '0974b291-264a-43d8-a350-04db1b3ede47' of volume 62f19c54-bca2-42c3-b76d-cd55e33e35d7 finished. {{(pid=129865) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 22 21:50:08.330365 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd4718fd0>, 'connection_properties': {'name': 'volumes/volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '62f19c54-bca2-42c3-b76d-cd55e33e35d7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd4594a90>}, 'force': True, ': True}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:50:08.694086 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] <== disconnect_volume: return (362ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:50:08.846892 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-d1c7b97c-aca3-428d-b7ee-18c15e21748c tempest-TestVolumeBackupRestore-1156611914 None] Create backup finished. backup: 0974b291-264a-43d8-a350-04db1b3ede47.
Jan 22 21:50:11.383984 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Restore backup started, backup: 0974b291-264a-43d8-a350-04db1b3ede47 volume: 7787920b-4db0-477d-a5ae-1db8121ceed7.
Jan 22 21:50:11.386971 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.100', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:50:11.388973 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.398121 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:50:11.398627 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc1cb212-b05a-460e-98db-0f84beca1f2d]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:cc5a3183ca8f\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:50:11.399947 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.405879 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e8609442-c6da-4492-a642-cf1d05d552a1]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:50:11.406899 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:50:11.408191 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8609442-c6da-4492-a642-cf1d05d552a1]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:50:11.408312 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.414628 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[9c9455e5-981d-410b-ba15-b62baaee8af1]: [Errno 2] No such file or directory: 'systool' {{(pid=164079) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]: Traceback (most recent call last):
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     ret = func(*f_args, **f_kwargs)
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     return func(*args, **kwargs)
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     return putils.execute(on_execute=on_execute,
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     obj = subprocess.Popen(cmd,
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 22 21:50:11.415257 ncdv-master-1 cinder-backup[129865]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 22 21:50:11.416869 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c9455e5-981d-410b-ba15-b62baaee8af1]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:50:11.417004 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.425727 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:50:11.426058 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa24498b-cb68-4151-965a-c369c72ae282]: (4, ('/dev/sda1\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:50:11.427165 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.435543 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:50:11.435874 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[942269bf-c527-4f06-aeed-4e6039bf5dbf]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:50:11.437756 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.444037 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=164079) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:50:11.444379 ncdv-master-1 cinder-backup[129865]: DEBUG oslo.privsep.daemon [-] privsep: reply[046c0a8e-6134-49b8-9df9-b39ed02a8488]: (4, ('3f9fc4cf-a152-254b-a00c-091ccc6f26f8\n', '')) {{(pid=164079) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 22 21:50:11.445177 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Running cmd (subprocess): nvme version {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 22 21:50:11.464505 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_concurrency.processutils [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=129865) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 22 21:50:11.465597 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.utils [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] <== get_connector_properties: return (78ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.100', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:cc5a3183ca8f', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '3f9fc4cf-a152-254b-a00c-091ccc6f26f8', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:e0b6af4b-807f-4ebe-9c72-0d0122d36aa8'} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:50:12.128359 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connector [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Factory for rbd on None {{(pid=129865) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 22 21:50:12.129886 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd45cbbe0>, 'connection_properties': {'name': 'volumes/volume-7787920b-4db0-477d-a5ae-1db8121ceed7', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7787920b-4db0-477d-a5ae-1db8121ceed7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:50:12.130964 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.linuxrbd [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] opening connection to ceph cluster (timeout=-1). {{(pid=129865) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 22 21:50:12.246713 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] <== connect_volume: return (116ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd45c90d0>} {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:50:12.254877 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:50:12.266255 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Starting restore from Ceph backup=0974b291-264a-43d8-a350-04db1b3ede47 to volume=volume-7787920b-4db0-477d-a5ae-1db8121ceed7 {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Jan 22 21:50:12.324378 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Looking for snapshot of backup base 'volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:50:12.324771 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Found snapshot 'backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:50:12.329769 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Volume file is RBD. {{(pid=129865) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Jan 22 21:50:12.358151 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Attempting differential restore. {{(pid=129865) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Jan 22 21:50:12.358925 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Attempting incremental restore from base='volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' snap='backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172' {{(pid=129865) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Jan 22 21:50:12.359259 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Performing differential transfer from 'volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' to 'volume-7787920b-4db0-477d-a5ae-1db8121ceed7' {{(pid=129865) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 22 21:50:12.359658 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47@backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172 -' into... {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 22 21:50:12.360026 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_wknukuql --pool volumes - volumes/volume-7787920b-4db0-477d-a5ae-1db8121ceed7' {{(pid=129865) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 22 21:50:55.320168 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Restore transfer completed in 42.9604s {{(pid=129865) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Jan 22 21:50:55.321145 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Jan 22 21:50:55.344438 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Restoring volume base metadata {{(pid=129865) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Jan 22 21:50:55.429008 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] No metadata of type 'volume-metadata' to restore {{(pid=129865) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 22 21:50:55.429382 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.driver [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Restoring volume glance metadata {{(pid=129865) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Jan 22 21:50:56.371820 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Restore to volume 7787920b-4db0-477d-a5ae-1db8121ceed7 finished successfully. {{(pid=129865) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Jan 22 21:50:56.373418 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fafd45cbbe0>, 'connection_properties': {'name': 'volumes/volume-7787920b-4db0-477d-a5ae-1db8121ceed7', 'hosts': ['10.222.0.100'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7787920b-4db0-477d-a5ae-1db8121ceed7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fafd45c90d0>}, 'force': True, ': False}" {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 22 21:50:56.465361 ncdv-master-1 cinder-backup[129865]: DEBUG os_brick.initiator.connectors.rbd [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] <== disconnect_volume: return (90ms) None {{(pid=129865) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 22 21:50:56.518557 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.manager [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Encryption key ID for volume 7787920b-4db0-477d-a5ae-1db8121ceed7 already matches encryption key ID in backup 0974b291-264a-43d8-a350-04db1b3ede47. {{(pid=129865) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Jan 22 21:50:56.711511 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:50:56.711778 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:50:56.734311 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-f76a8884-136f-43ce-b33b-b0e47ab86a10 tempest-TestVolumeBackupRestore-1156611914 None] Finished restoring backup 0974b291-264a-43d8-a350-04db1b3ede47 to volume 7787920b-4db0-477d-a5ae-1db8121ceed7.
Jan 22 21:51:56.716790 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:51:56.717482 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:52:56.721492 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:52:56.722209 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:53:56.717625 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:53:56.718336 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:54:56.722266 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:54:56.722951 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:55:56.718325 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:55:56.719034 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:56:15.395425 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Delete backup started, backup: 0974b291-264a-43d8-a350-04db1b3ede47.
Jan 22 21:56:15.397132 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Delete started for backup=0974b291-264a-43d8-a350-04db1b3ede47 {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 22 21:56:15.397711 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Trying diff format basename='volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' for backup base image of volume 62f19c54-bca2-42c3-b76d-cd55e33e35d7. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 22 21:56:15.711990 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Looking for snapshot of backup base 'volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 22 21:56:15.712364 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Found snapshot 'backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172' {{(pid=129865) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 22 21:56:15.712737 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Deleting backup snapshot='backup.0974b291-264a-43d8-a350-04db1b3ede47.snap.1642888164.2860172' {{(pid=129865) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 22 21:56:18.251724 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Deleting backup base image='volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' of volume 62f19c54-bca2-42c3-b76d-cd55e33e35d7.
Jan 22 21:56:23.200114 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_db.sqlalchemy.engines [None req-adda2dbc-3a9f-4211-952f-169b4cb82538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 22 21:56:29.042341 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Base backup image='volume-62f19c54-bca2-42c3-b76d-cd55e33e35d7.backup.0974b291-264a-43d8-a350-04db1b3ede47' of volume 62f19c54-bca2-42c3-b76d-cd55e33e35d7 deleted. {{(pid=129865) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 22 21:56:29.192656 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.backup.drivers.ceph [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Delete of backup '0974b291-264a-43d8-a350-04db1b3ede47' for volume '62f19c54-bca2-42c3-b76d-cd55e33e35d7' finished. {{(pid=129865) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 22 21:56:29.287375 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.quota [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Created reservations ['1af11166-a830-47ca-8ea2-c8e4b73b7629', '33d12517-df11-4648-8ea5-09580ef30732'] {{(pid=129865) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 22 21:56:29.618434 ncdv-master-1 cinder-backup[129865]: INFO cinder.backup.manager [None req-70df9170-aba2-4332-98e3-dd92f0ced405 tempest-TestVolumeBackupRestore-1156611914 None] Delete backup finished, backup 0974b291-264a-43d8-a350-04db1b3ede47 deleted.
Jan 22 21:56:56.715148 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:56:56.715809 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:57:56.711382 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:57:56.712070 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:58:56.709273 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:58:56.710027 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 21:59:56.706446 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 21:59:56.707095 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:00:56.702997 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:00:56.703717 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:01:56.698887 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:01:56.700179 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:02:56.695571 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:02:56.696260 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:03:56.692244 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:03:56.692879 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:04:56.697494 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:04:56.698290 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:05:56.702160 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:05:56.702874 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:06:56.707631 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:06:56.708244 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:07:56.703261 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:07:56.703892 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:08:56.708498 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:08:56.709280 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:09:56.713686 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:09:56.714402 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:10:56.711251 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:10:56.711929 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:11:56.707233 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:11:56.707907 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:12:56.703956 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:12:56.704624 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:13:56.700866 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:13:56.702158 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:14:56.707118 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:14:56.707743 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:15:56.711564 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:15:56.712225 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:16:56.707568 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:16:56.708271 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:17:56.712905 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:17:56.713598 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:19:56.719750 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:19:56.720384 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:20:56.724711 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:20:56.725346 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:21:56.730222 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:21:56.730922 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:22:56.726386 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:22:56.727066 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:23:56.722642 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:23:56.723266 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:24:56.728082 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:24:56.728757 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:25:56.732810 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:25:56.733468 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:26:56.738037 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:26:56.745342 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:27:56.743507 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:27:56.744217 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:28:56.749205 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:28:56.749897 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:29:56.754270 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:29:56.755482 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:30:56.760067 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:30:56.760742 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:31:56.765333 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:31:56.766020 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:32:56.771505 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:32:56.772331 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:33:56.767232 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:33:56.767889 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:34:56.772403 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:34:56.773145 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:35:56.769053 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:35:56.770013 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:36:56.765426 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:36:56.766152 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:37:56.761703 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:37:56.762397 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:38:56.767718 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:38:56.768384 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:39:56.773101 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:39:56.773824 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:40:56.778270 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:40:56.778978 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:41:56.784263 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:41:56.784979 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:42:56.789999 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:42:56.790677 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:43:56.786332 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:43:56.787096 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:44:56.783768 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:44:56.784455 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:45:56.779715 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:45:56.780393 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:46:56.784280 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:46:56.790496 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:47:56.788854 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:47:56.789536 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:48:56.785221 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:48:56.785927 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 22 22:49:56.790164 ncdv-master-1 cinder-backup[129865]: DEBUG oslo_service.periodic_task [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129865) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 22 22:49:56.791060 ncdv-master-1 cinder-backup[129865]: DEBUG cinder.manager [None req-9a3be5b3-309a-40fd-8869-f27b5a393a0f None None] Notifying Schedulers of capabilities ... {{(pid=129865) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}