-- Logs begin at Sat 2021-11-27 20:01:15 UTC, end at Sat 2021-11-27 22:28:21 UTC. -- Nov 27 20:36:46.291032 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service. Nov 27 20:36:49.605637 ncdv-master-1 cinder-backup[128556]: INFO cinder.cmd.backup [-] Backup running with 1 processes. Nov 27 20:36:49.607463 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=128556) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Nov 27 20:36:49.608011 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=128556) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Nov 27 20:36:49.761462 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_db.sqlalchemy.engines [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=128556) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 20:36:49.986578 ncdv-master-1 cinder-backup[128556]: INFO cinder.rpc [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Nov 27 20:36:49.990741 ncdv-master-1 cinder-backup[128556]: INFO cinder.rpc [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version. Nov 27 20:36:49.995059 ncdv-master-1 cinder-backup[128556]: INFO cinder.rpc [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Nov 27 20:36:49.999329 ncdv-master-1 cinder-backup[128556]: INFO cinder.rpc [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version. Nov 27 20:36:50.004634 ncdv-master-1 cinder-backup[128556]: INFO cinder.rpc [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Nov 27 20:36:50.008700 ncdv-master-1 cinder-backup[128556]: INFO cinder.rpc [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Nov 27 20:36:50.057788 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_db.sqlalchemy.engines [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=128556) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 20:36:50.080886 ncdv-master-1 cinder-backup[128556]: INFO oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Starting 1 workers Nov 27 20:36:50.086923 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Started child 129001 {{(pid=128556) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Nov 27 20:36:50.088217 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Full set of CONF: {{(pid=128556) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Nov 27 20:36:50.088814 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] ******************************************************************************** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 27 20:36:50.089295 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] Configuration options gathered from: {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 27 20:36:50.090050 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 27 20:36:50.091392 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 27 20:36:50.092090 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] ================================================================================ {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 27 20:36:50.093115 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] allow_availability_zone_fallback = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.093905 ncdv-master-1 cinder-backup[129001]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0) Nov 27 20:36:50.095078 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] allow_compression_on_image_upload = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.096080 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] allowed_direct_url_schemes = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.096977 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] api_rate_limit = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.097590 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] auth_strategy = keystone {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.098187 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_availability_zone = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.098769 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_api_class = cinder.backup.api.API {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.099432 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_chunk_size = 134217728 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.100053 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_conf = /etc/ceph/ceph.conf {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.100574 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_image_journals = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.101114 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_pool = backups {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.101658 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_stripe_count = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.102252 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_stripe_unit = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.102781 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_ceph_user = cinder-bak {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.103504 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_driver = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.104071 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_driver_init_check_interval = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.104585 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_driver_stats_polling_interval = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.105896 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.106472 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_max_operations = 15 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.107051 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_metadata_version = 2 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.107751 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_name_template = backup-%s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.108254 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_native_threads_pool_size = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.108804 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_object_number_per_notification = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.109302 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_service_inithost_offload = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.109815 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_timer_interval = 120 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.110342 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_use_same_host = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.110864 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_use_temp_snapshot = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.111443 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backup_workers = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.111977 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] chap_password = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.112489 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] chap_username = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.113017 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.113785 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_internal_tenant_project_id = 59c371d9310548a3ac5cf1cb274d0e34 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.114412 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_internal_tenant_user_id = 198c9372013440ec942a0192a73a231d {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.115317 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] compression_format = gzip {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.115909 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] config_dir = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.116358 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.116890 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] config_source = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.117423 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.117972 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] control_exchange = openstack {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.118539 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] debug = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.119032 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] default_availability_zone = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.119571 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] default_group_type = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.120397 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.122949 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] default_volume_type = ceph {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.123967 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] driver_client_cert = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.125454 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] driver_client_cert_key = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.126248 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] driver_data_namespace = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.126992 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] driver_ssl_cert_path = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.127768 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] driver_ssl_cert_verify = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.128354 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] driver_use_ssl = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.128826 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] enable_new_services = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.129376 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] enable_unsupported_driver = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.129943 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] enabled_backends = ['ceph'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.130579 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] enforce_multipath_for_image_xfer = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.131081 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] filter_function = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.131626 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_api_insecure = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.132175 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_api_servers = ['http://10.222.0.21/image'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.132676 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_api_ssl_compression = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.133299 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_ca_certificates_file = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.133950 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_catalog_info = image:glance:publicURL {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.134601 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.135725 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.136271 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.136812 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_num_retries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.137320 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] glance_request_timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.137835 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] goodness_function = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.138363 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] graceful_shutdown_timeout = 5 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.138910 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] group_api_class = cinder.group.api.API {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.139473 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] host = ncdv-master-1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.140043 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iet_conf = /etc/iet/ietd.conf {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.140589 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_compress_on_upload = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.141100 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_conversion_address_space_limit = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.141613 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_conversion_cpu_limit = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.142239 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.142775 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_upload_use_cinder_backend = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.143307 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_upload_use_internal_tenant = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.143824 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_volume_cache_enabled = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.144381 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_volume_cache_max_count = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.144885 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] image_volume_cache_max_size_gb = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.145782 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] instance_format = [instance: %(uuid)s] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.146334 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.146866 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iscsi_iotype = fileio {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.147527 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iscsi_secondary_ip_addresses = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.148074 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iscsi_target_flags = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.148689 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iscsi_write_cache = on {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.149568 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iser_helper = tgtadm {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.150896 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iser_ip_address = 10.222.0.21 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.151495 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iser_port = 3260 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.152070 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.152607 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_config_append = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.153154 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.153679 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_dir = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.154197 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_file = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.154787 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_options = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.155963 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_rotate_interval = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.156606 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_rotate_interval_type = days {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.157217 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] log_rotation_type = none {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.157796 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.158297 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.158994 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.159540 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.160008 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.160575 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] max_age = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.161151 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] max_logfile_count = 30 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.161684 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] max_logfile_size_mb = 200 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.162234 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] max_over_subscription_ratio = 20.0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.162797 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] message_reap_interval = 86400 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.163392 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] message_ttl = 2592000 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.163891 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] monkey_patch = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.164503 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] monkey_patch_modules = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.164945 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] my_ip = 10.222.0.21 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.165659 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] no_snapshot_gb_quota = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.166380 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] num_iser_scan_tries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.166936 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] num_shell_tries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.167502 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] num_volume_device_scan_tries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.168545 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] nvmet_ns_id = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.169049 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] nvmet_port_id = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.169454 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.lockutils [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] Lock "cleanup_incomplete_backups_128556" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.003s {{(pid=129001) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Nov 27 20:36:50.169786 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_max_limit = 1000 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.170299 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_volume_ext_list = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.170761 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] Cleaning up incomplete backup operations. Nov 27 20:36:50.171080 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.171645 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_volume_listen = 0.0.0.0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.172231 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_volume_listen_port = 8776 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.172719 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_volume_use_ssl = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.173390 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] osapi_volume_workers = 2 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.173943 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] per_volume_size_limit = -1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.174493 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] periodic_fuzzy_delay = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.175153 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] periodic_interval = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.176295 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] public_endpoint = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.176882 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] publish_errors = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.177481 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_backup_gigabytes = 1000 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.178086 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_backups = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.178634 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_consistencygroups = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.180004 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.180573 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_gigabytes = 1000 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.181132 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_groups = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.181816 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_snapshots = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.182335 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] quota_volumes = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.182843 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] rate_limit_burst = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.184147 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] rate_limit_except_level = CRITICAL {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.184771 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] rate_limit_interval = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.185323 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] replication_device = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.186987 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] report_discard_supported = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.187545 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] report_interval = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.188198 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] reservation_clean_interval = 86400 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.188645 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] reservation_expire = 86400 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.189156 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] reserved_percentage = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.190075 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.191316 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] restore_discard_excess_bytes = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.191857 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.192440 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] rpc_response_timeout = 120 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.192943 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] run_external_periodic_tasks = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.193461 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.194039 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] scst_target_driver = iscsi {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.194568 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] scst_target_iqn_name = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.195024 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_down_time = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.195666 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] snapshot_name_template = snapshot-%s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.196574 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] split_loggers = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.197239 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] state_path = /opt/stack/data/cinder {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.197886 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] storage_availability_zone = nova {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.198447 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] storage_protocol = iscsi {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.199066 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] syslog_log_facility = LOG_USER {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.199805 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] target_helper = lioadm {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.200458 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] target_ip_address = 10.222.0.21 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.201019 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] target_port = 3260 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.201649 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.202208 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] target_protocol = iscsi {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.202721 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] trace_flags = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.203947 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] transfer_api_class = cinder.transfer.api.API {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.204545 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] transport_url = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.205114 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] until_refresh = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.205631 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_chap_auth = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.206359 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_default_quota_class = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.207131 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_eventlog = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.207821 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_journal = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.208421 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_json = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.208943 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_multipath_for_image_xfer = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.210580 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_stderr = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.211187 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] use_syslog = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.211866 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] verify_glance_signatures = enabled {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.212461 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_api_class = cinder.volume.api.API {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.212993 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_backend_name = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.213530 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_clear = zero {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.214027 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_clear_ionice = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.214568 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_clear_size = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.215324 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.215937 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_copy_bps_limit = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.223759 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_dd_blocksize = 1M {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.224523 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.225234 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_name_template = volume-%s {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.226093 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volume_usage_audit_period = month {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.227134 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.227725 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] watch_log_file = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 27 20:36:50.230989 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.231713 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.232284 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.enforce_new_defaults = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.232867 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.enforce_scope = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.233543 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.policy_default_rule = default {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.234251 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.234826 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.policy_file = policy.yaml {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.235678 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.236366 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.237072 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.237586 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.238093 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.238654 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_concurrency.disable_process_locking = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.239333 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.240072 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.admin_password = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.240966 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.admin_tenant_name = admin {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.241498 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.admin_token = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.242076 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.admin_user = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.242686 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_admin_prefix = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.243282 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.244190 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_port = 35357 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.245109 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_protocol = https {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.245656 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_section = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.247044 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_type = password {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.247604 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_uri = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.248129 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.auth_version = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.248646 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.cache = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.249489 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.250491 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.251063 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.delay_auth_decision = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.251657 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.252244 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.http_connect_timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.252777 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.253317 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.identity_uri = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.253927 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.include_service_catalog = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.254637 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.insecure = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.255175 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.interface = public {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.255720 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.256228 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.256795 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.257402 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.257939 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.258527 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.259091 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_secret_key = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.259505 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_security_strategy = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.260114 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.260450 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.260964 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.region_name = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263360 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263360 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.service_token_roles_required = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263360 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.service_type = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263360 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.token_cache_time = 300 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263360 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] keystone_authtoken.www_authenticate_uri = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263360 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.backend = dogpile.cache.null {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.263786 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.backend_argument = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.264171 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.config_prefix = cache.oslo {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.264668 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.debug_cache_backend = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.265086 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.enabled = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.265481 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.expiration_time = 600 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.265846 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_dead_retry = 300 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.266240 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.266615 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.266987 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_pool_maxsize = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.267505 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_pool_unused_timeout = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.267950 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_servers = ['localhost:11211'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.268359 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.memcache_socket_timeout = 1.0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.268749 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.proxies = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.269151 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.tls_allowed_ciphers = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.269501 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.tls_cafile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.269848 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.tls_certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.270211 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.tls_enabled = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.270646 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cache.tls_keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.271060 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] coordination.backend_url = etcd3+http://10.222.0.21:2379 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.271521 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.backend = sqlalchemy {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.271835 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.connection = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.272250 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.connection_debug = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.272621 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.connection_parameters = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.273024 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.connection_recycle_time = 3600 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.273415 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.connection_trace = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.273771 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.db_inc_retry_interval = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.274240 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.db_max_retries = 20 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.274726 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.db_max_retry_interval = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.275319 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.db_retry_interval = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.275715 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.max_overflow = 50 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.276119 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.max_pool_size = 5 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.276495 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.max_retries = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.276853 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.mysql_enable_ndb = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.277399 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.mysql_sql_mode = TRADITIONAL {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.277781 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.pool_timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.278167 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.retry_interval = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.278537 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.slave_connection = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.278891 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.sqlite_synchronous = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.279273 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] database.use_db_reconnect = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.279675 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.280045 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.280658 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_metrics.metrics_process_name = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.281029 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.281387 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.281800 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.connection_string = messaging:// {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.282111 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.enabled = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.282478 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.es_doc_type = notification {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.282840 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.es_scroll_size = 10000 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.283210 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.es_scroll_time = 2m {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.283586 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.filter_error_trace = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.283951 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.hmac_keys = SECRET_KEY {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.284425 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.sentinel_service_name = mymaster {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.284921 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.socket_timeout = 0.1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.285324 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] profiler.trace_sqlalchemy = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.285738 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] privsep_osbrick.capabilities = [21] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.286114 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] privsep_osbrick.group = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.286482 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] privsep_osbrick.helper_command = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.286837 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.287190 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] privsep_osbrick.thread_pool_size = 4 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.287670 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] privsep_osbrick.user = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.288079 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.288485 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_sys_admin.group = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.288832 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_sys_admin.helper_command = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.289192 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.289539 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.289890 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] cinder_sys_admin.user = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.290247 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.auth_section = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.290701 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.auth_type = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.291055 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.cafile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.291429 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.291806 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.collect_timing = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.292164 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.insecure = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.292515 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.293633 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.send_service_user_token = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.293997 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.split_loggers = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.294472 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] service_user.timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.295027 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.backend_availability_zone = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.295424 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.chap_password = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.295811 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.chap_username = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.296178 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.296598 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.296974 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.297319 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.driver_client_cert = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.297777 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.driver_client_cert_key = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.298134 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.driver_data_namespace = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.298478 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.driver_ssl_cert_path = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.298849 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.299198 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.driver_use_ssl = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.299572 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.enable_deferred_deletion = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.299922 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.enable_unsupported_driver = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.300277 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.300725 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.filter_function = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.301325 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.goodness_function = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.301725 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.302079 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.302431 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.302781 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.image_volume_cache_enabled = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.303131 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.303506 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.303868 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iscsi_iotype = fileio {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.304226 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.304700 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iscsi_target_flags = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.305188 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iscsi_write_cache = on {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.305561 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iser_helper = tgtadm {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.306137 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iser_ip_address = 10.222.0.21 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.306492 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iser_port = 3260 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.306861 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.307261 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.307735 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.308143 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.num_shell_tries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.309000 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.309354 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.nvmet_ns_id = 10 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.309750 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.nvmet_port_id = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.310128 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rados_connect_timeout = -1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.310477 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rados_connection_interval = 5 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.310931 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rados_connection_retries = 3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.311542 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_ceph_conf = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.311924 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_cluster_name = ceph {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.312278 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.312929 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.313284 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.313655 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_pool = rbd {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.314000 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_secret_uuid = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.314349 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.315246 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.rbd_user = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.315643 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.replication_connect_timeout = 5 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.315994 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.replication_device = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.316345 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.report_discard_supported = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.316958 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.317120 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.reserved_percentage = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.317457 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.scst_target_driver = iscsi {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.317838 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.scst_target_iqn_name = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.318218 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.storage_protocol = iscsi {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.318588 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.target_helper = tgtadm {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.319054 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.target_ip_address = 10.222.0.21 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.319456 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.target_port = 3260 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.319825 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.320190 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.target_protocol = iscsi {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.320536 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.trace_flags = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.320997 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.unique_fqdn_network = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.321349 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.use_chap_auth = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.321939 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.322494 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_backend_name = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.323128 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_clear = zero {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.323510 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_clear_ionice = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.323925 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_clear_size = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.324294 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.324753 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.325465 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.325859 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.326263 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_reports.file_event_handler = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.326727 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_reports.file_event_handler_interval = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.327020 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_reports.log_dir = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.327488 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.328043 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.328357 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.328778 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.329161 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.329517 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.329891 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.330243 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.330595 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.331300 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.331956 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.332333 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.332711 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.333076 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.333431 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.333799 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.334151 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.334505 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.334966 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.335377 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.336019 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.336414 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.ssl = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.336800 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.337172 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.337543 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.338010 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_rabbit.ssl_version = {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.338431 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_notifications.driver = [] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.338790 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_notifications.retry = -1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.339199 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.339832 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] oslo_messaging_notifications.transport_url = **** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.340267 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.340696 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.341173 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.barbican_api_version = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.341590 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.barbican_endpoint = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.342234 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.barbican_endpoint_type = public {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.342604 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.barbican_region_name = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.342954 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.cafile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.343318 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.343691 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.collect_timing = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.344039 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.insecure = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.344410 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.344905 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.number_of_retries = 60 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.345261 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.retry_delay = 1 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.345776 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.send_service_user_token = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.346142 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.split_loggers = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.346491 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.346841 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.verify_ssl = True {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.347441 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican.verify_ssl_path = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.347804 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.auth_section = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.348288 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.auth_type = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.348629 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.cafile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.348971 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.349317 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.collect_timing = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.349685 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.insecure = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.350029 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.350377 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.split_loggers = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.350720 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] barbican_service_user.timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.351109 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.approle_role_id = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.351574 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.approle_secret_id = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.352150 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.cafile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.352512 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.certfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.352864 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.collect_timing = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.353215 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.insecure = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.353844 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.keyfile = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.354226 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.kv_mountpoint = secret {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.354997 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.kv_version = 2 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.356040 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.namespace = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.356392 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.root_token_id = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.356752 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.split_loggers = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.357101 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.ssl_ca_crt_file = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.357448 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.timeout = None {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.357851 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.use_ssl = False {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.358340 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 27 20:36:50.359016 ncdv-master-1 cinder-backup[128556]: DEBUG oslo_service.service [None req-ec3b718f-5401-4b82-b435-7890c67ce7df None None] ******************************************************************************** {{(pid=128556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 27 20:36:50.387823 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 20:36:50.415890 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.lockutils [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] Lock "cleanup_incomplete_backups_128556" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.248s {{(pid=129001) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Nov 27 20:36:50.424697 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 20:36:50.431229 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:36:50.461443 ncdv-master-1 cinder-backup[129001]: INFO cinder.keymgr.migration [None req-e6be8431-3f19-4590-8029-bc2832b5f36f None None] Not migrating encryption keys because the ConfKeyManager is still in use. Nov 27 20:36:50.488575 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.service [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] Creating RPC server for service cinder-backup {{(pid=129001) start /opt/stack/cinder/cinder/service.py:231}} Nov 27 20:36:50.497633 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.service [None req-991707b1-bd33-41ec-b8d9-a10de15fea8c None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=129001) start /opt/stack/cinder/cinder/service.py:237}} Nov 27 20:38:41.533983 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:38:41.534712 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:39:41.530811 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:39:41.531534 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:40:41.536459 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:40:41.537118 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:41:41.541862 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:41:41.542461 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:42:41.546596 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:42:41.547624 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:43:41.552339 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:43:41.553039 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:44:41.548522 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:44:41.549145 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:45:41.555174 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:45:41.556881 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:46:41.553096 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:46:41.553827 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:47:41.558314 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:47:41.560038 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:48:41.562749 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:48:41.563492 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:49:41.568383 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:49:41.569066 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:50:41.573082 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:50:41.573749 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:51:41.578030 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:51:41.578672 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:52:41.578941 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:52:41.579638 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:53:41.578202 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:53:41.578816 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:54:41.579990 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:54:41.580700 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:55:41.581076 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:55:41.581696 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:56:41.586171 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:56:41.586788 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:57:41.583561 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:57:41.584244 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:58:41.584124 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:58:41.584820 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 20:59:41.580076 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 20:59:41.580815 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:00:41.578098 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:00:41.579212 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:01:41.576955 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:01:41.577587 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:02:41.574728 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:02:41.575527 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:03:41.570334 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:03:41.571012 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:04:41.567405 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:04:41.568053 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:05:41.563977 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:05:41.564560 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:06:41.565628 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:06:41.566727 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:07:41.564442 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:07:41.565064 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:08:41.569956 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:08:41.570539 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:09:41.567318 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:09:41.567962 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:10:41.568185 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:10:41.568751 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:11:41.573073 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:11:41.573641 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:12:41.569452 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:12:41.570583 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:13:41.574401 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:13:41.575186 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:13:48.155960 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Create backup started, backup: 3997df94-7bf1-415e-ae47-e84b32d51bb3 volume: bc95af6a-3bbf-4795-81b0-cda182d7053e. Nov 27 21:13:48.179955 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:13:48.183142 ncdv-master-1 cinder-backup[129001]: INFO oslo.privsep.daemon [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpebvqz5n4/privsep.sock'] Nov 27 21:13:48.351427 ncdv-master-1 sudo[164845]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpebvqz5n4/privsep.sock Nov 27 21:13:48.364345 ncdv-master-1 sudo[164845]: pam_unix(sudo:session): session opened for user root by (uid=0) Nov 27 21:13:50.798837 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-77d2af62-f14e-4e15-b4e0-2fb0b29d193d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:13:52.068933 ncdv-master-1 sudo[164845]: pam_unix(sudo:session): session closed for user root Nov 27 21:13:52.073277 ncdv-master-1 cinder-backup[129001]: INFO oslo.privsep.daemon [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Spawned new privsep daemon via rootwrap Nov 27 21:13:52.073816 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Accepted privsep connection to /tmp/tmpebvqz5n4/privsep.sock {{(pid=129001) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}} Nov 27 21:13:52.075453 ncdv-master-1 cinder-backup[129001]: INFO oslo.privsep.daemon [-] privsep daemon starting Nov 27 21:13:52.075793 ncdv-master-1 cinder-backup[129001]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Nov 27 21:13:52.076497 ncdv-master-1 cinder-backup[129001]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none Nov 27 21:13:52.076982 ncdv-master-1 cinder-backup[129001]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 165020 Nov 27 21:13:52.079121 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[56fbf9a1-7312-4997-9ed5-788bcfdae4e2]: (2,) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:52.727334 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:52.778781 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.052s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:13:52.779372 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[9efac92d-e70f-4220-a621-31f4a44b5e87]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:52.781212 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:52.801286 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b56cb09b-8c66-4608-bc95-8f952cbafc9f]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:13:52.821601 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:13:52.823522 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[b56cb09b-8c66-4608-bc95-8f952cbafc9f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:52.827536 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:52.841835 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[bc7c8a8b-6325-4460-906c-76918496b3c4]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:13:52.842551 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:13:52.844970 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc7c8a8b-6325-4460-906c-76918496b3c4]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:52.850076 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:52.902666 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.053s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:13:52.904540 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[08159bc9-70aa-4dfb-b6c7-dbc5f5d3a5b3]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:52.907917 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:53.008288 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.102s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:13:53.008779 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[0952a4fc-9568-4807-a53c-84a61c67f5fe]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:53.010364 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:53.045044 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.034s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:13:53.046623 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[0814b677-ca41-4a24-abe2-f45f8ef838b5]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:13:53.048751 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:13:53.112465 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] CMD "nvme version" returned: 0 in 0.064s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:13:53.137749 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] <== get_connector_properties: return (4954ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:13:54.336277 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:13:54.339119 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-bc95af6a-3bbf-4795-81b0-cda182d7053e', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'bc95af6a-3bbf-4795-81b0-cda182d7053e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:13:54.341986 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:13:54.505742 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] <== connect_volume: return (162ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:13:54.556899 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Starting backup of volume='bc95af6a-3bbf-4795-81b0-cda182d7053e'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:13:54.558123 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Volume file is RBD: attempting optimized backup {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 27 21:13:54.666335 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Creating base image 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' {{(pid=129001) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 27 21:13:54.667606 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:13:56.504795 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Using --from-snap 'None' for incremental backup of volume bc95af6a-3bbf-4795-81b0-cda182d7053e. {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 27 21:13:56.505459 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Creating backup snapshot='backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182' {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 27 21:13:57.486589 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Performing differential transfer from 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e' to 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:13:57.487000 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_ttd2djxk --pool volumes volumes/volume-bc95af6a-3bbf-4795-81b0-cda182d7053e@backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:13:57.487432 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:13:58.529190 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Differential backup transfer completed in 1.0416s {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 27 21:13:58.529874 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:13:58.544588 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:13:58.545285 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:13:58.558149 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:13:58.558723 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:13:58.568892 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] No metadata type 'volume-glance-metadata' available {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 27 21:13:58.569593 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Backing up metadata for volume bc95af6a-3bbf-4795-81b0-cda182d7053e. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:13:58.618398 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Backup '3997df94-7bf1-415e-ae47-e84b32d51bb3' of volume bc95af6a-3bbf-4795-81b0-cda182d7053e finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:13:58.619891 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-bc95af6a-3bbf-4795-81b0-cda182d7053e', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'bc95af6a-3bbf-4795-81b0-cda182d7053e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:13:58.649417 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] <== disconnect_volume: return (28ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:13:58.726669 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-05f6bba3-2ff6-460f-aabf-dc7ae88e92cc tempest-VolumesBackupsAdminTest-1903881971 None] Create backup finished. backup: 3997df94-7bf1-415e-ae47-e84b32d51bb3. Nov 27 21:13:59.802111 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-bb8580fc-ebe2-4bdb-b805-d7071cad556f tempest-VolumesBackupsAdminTest-1080464940 None] Export record started, backup: 3997df94-7bf1-415e-ae47-e84b32d51bb3. Nov 27 21:13:59.805116 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-bb8580fc-ebe2-4bdb-b805-d7071cad556f tempest-VolumesBackupsAdminTest-1080464940 None] Export record finished, backup 3997df94-7bf1-415e-ae47-e84b32d51bb3 exported. Nov 27 21:13:59.963718 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-dc55a770-4645-474f-b14f-35151115ebf1 tempest-VolumesBackupsAdminTest-1080464940 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiMzQwNDE2ODktODA2ZC00OWM2LTk4ZmYtNTFmNDUyYzdiZjBiIiwgInVzZXJfaWQiOiAiNDcwMTc0Yzg2NWM1NDdlNDhhNTU3NTEwMTVjNjk5MzEiLCAicHJvamVjdF9pZCI6ICJjMzQ2NjE3ZTdhYTQ0ODI4YWYzNDlkOTFiZDI0ODdlYiIsICJ2b2x1bWVfaWQiOiAiYmM5NWFmNmEtM2JiZi00Nzk1LTgxYjAtY2RhMTgyZDcwNTNlIiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC00MDgyNzI5MjAiLCAiZGlzcGxheV9kZXNjcmlwdGlvbiI6IG51bGwsICJzZXJ2aWNlX21ldGFkYXRhIjogIntcImJhc2VcIjogXCJ2b2x1bWUtYmM5NWFmNmEtM2JiZi00Nzk1LTgxYjAtY2RhMTgyZDcwNTNlLmJhY2t1cC4zOTk3ZGY5NC03YmYxLTQxNWUtYWU0Ny1lODRiMzJkNTFiYjNcIn0iLCAic2VydmljZSI6ICJjaW5kZXIuYmFja3VwLmRyaXZlcnMuY2VwaC5DZXBoQmFja3VwRHJpdmVyIiwgIm9iamVjdF9jb3VudCI6IDAsICJ0ZW1wX3ZvbHVtZV9pZCI6IG51bGwsICJ0ZW1wX3NuYXBzaG90X2lkIjogbnVsbCwgIm51bV9kZXBlbmRlbnRfYmFja3VwcyI6IDAsICJzbmFwc2hvdF9pZCI6IG51bGwsICJkYXRhX3RpbWVzdGFtcCI6ICIyMDIxLTExLTI3VDIxOjEzOjQ4WiIsICJyZXN0b3JlX3ZvbHVtZV9pZCI6IG51bGwsICJtZXRhZGF0YSI6IHt9LCAiZW5jcnlwdGlvbl9rZXlfaWQiOiBudWxsLCAiY3JlYXRlZF9hdCI6ICIyMDIxLTExLTI3VDIxOjEzOjQ4WiIsICJ1cGRhdGVkX2F0IjogIjIwMjEtMTEtMjdUMjE6MTM6NTlaIiwgImRlbGV0ZWRfYXQiOiBudWxsLCAiZGVsZXRlZCI6IGZhbHNlfQ==. Nov 27 21:13:59.971302 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-dc55a770-4645-474f-b14f-35151115ebf1 tempest-VolumesBackupsAdminTest-1080464940 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:14:00.014712 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-dc55a770-4645-474f-b14f-35151115ebf1 tempest-VolumesBackupsAdminTest-1080464940 None] Import record id 34041689-806d-49c6-98ff-51f452c7bf0b metadata from driver finished. Nov 27 21:14:02.315909 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Restore backup started, backup: 3997df94-7bf1-415e-ae47-e84b32d51bb3 volume: 0d9674cb-0c68-4cf7-85a9-1a4f8311055a. Nov 27 21:14:02.327768 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:14:02.329291 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.337334 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:02.337893 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[a82d2fdd-11bb-4b31-a550-c5f2e96923da]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:02.339217 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.344800 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[00f0724d-4731-48d5-962b-7625855b8077]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:14:02.345373 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:14:02.346794 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[00f0724d-4731-48d5-962b-7625855b8077]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:02.347036 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.352492 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f8d58ef9-f4dd-469b-b2fe-5d621da3723d]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:14:02.353067 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:14:02.354436 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8d58ef9-f4dd-469b-b2fe-5d621da3723d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:02.354806 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.362973 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:02.363394 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[b32c59f9-9d25-41bf-bebf-f9672768e18d]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:02.364428 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.374505 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:02.374895 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[d74f6469-34fb-4796-ae83-1c33133ef123]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:02.376288 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.388615 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.012s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:02.388947 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a595909-7a34-4076-a85e-f9c3b74b6061]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:02.389800 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:02.408489 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] CMD "nvme version" returned: 0 in 0.017s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:02.409577 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] <== get_connector_properties: return (81ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:14:03.089035 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:14:03.090940 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-0d9674cb-0c68-4cf7-85a9-1a4f8311055a', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '0d9674cb-0c68-4cf7-85a9-1a4f8311055a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:14:03.092136 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:14:03.156554 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] <== connect_volume: return (65ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:14:03.165458 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:14:03.176250 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Starting restore from Ceph backup=3997df94-7bf1-415e-ae47-e84b32d51bb3 to volume=volume-0d9674cb-0c68-4cf7-85a9-1a4f8311055a {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 27 21:14:03.249548 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Looking for snapshot of backup base 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:14:03.249991 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Found snapshot 'backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:14:03.254676 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Volume file is RBD. {{(pid=129001) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 27 21:14:03.290996 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Attempting differential restore. {{(pid=129001) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 27 21:14:03.291492 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Attempting incremental restore from base='volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' snap='backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182' {{(pid=129001) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 27 21:14:03.291895 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Performing differential transfer from 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' to 'volume-0d9674cb-0c68-4cf7-85a9-1a4f8311055a' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:14:03.292307 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3@backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:14:03.292685 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_i1niusv3 --pool volumes - volumes/volume-0d9674cb-0c68-4cf7-85a9-1a4f8311055a' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:14:04.439968 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Restore transfer completed in 1.1474s {{(pid=129001) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 27 21:14:04.441027 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 27 21:14:04.467030 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Restoring volume base metadata {{(pid=129001) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 27 21:14:04.489859 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] No metadata of type 'volume-metadata' to restore {{(pid=129001) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 27 21:14:04.490364 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=129001) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 27 21:14:04.494872 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Restore to volume 0d9674cb-0c68-4cf7-85a9-1a4f8311055a finished successfully. {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 27 21:14:04.496464 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-0d9674cb-0c68-4cf7-85a9-1a4f8311055a', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '0d9674cb-0c68-4cf7-85a9-1a4f8311055a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:14:04.506338 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] <== disconnect_volume: return (9ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:14:04.551865 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.manager [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Encryption key ID for volume 0d9674cb-0c68-4cf7-85a9-1a4f8311055a already matches encryption key ID in backup 3997df94-7bf1-415e-ae47-e84b32d51bb3. {{(pid=129001) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 27 21:14:04.584924 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-0ce1a5f6-918b-426e-bad3-bd4a014c6bee tempest-VolumesBackupsAdminTest-1903881971 None] Finished restoring backup 3997df94-7bf1-415e-ae47-e84b32d51bb3 to volume 0d9674cb-0c68-4cf7-85a9-1a4f8311055a. Nov 27 21:14:05.650594 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Delete backup started, backup: 34041689-806d-49c6-98ff-51f452c7bf0b. Nov 27 21:14:05.653228 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Delete started for backup=34041689-806d-49c6-98ff-51f452c7bf0b {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:14:05.653856 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Trying diff format basename='volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' for backup base image of volume bc95af6a-3bbf-4795-81b0-cda182d7053e. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:14:05.715565 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Looking for snapshot of backup base 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:14:05.716464 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Backup '34041689-806d-49c6-98ff-51f452c7bf0b' has no snapshot {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}} Nov 27 21:14:05.717234 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] No backup snapshot to delete {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Nov 27 21:14:05.727563 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Backup base image of volume bc95af6a-3bbf-4795-81b0-cda182d7053e still has 1 snapshots so skipping base image delete. Nov 27 21:14:05.768430 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Metadata backup object 'backup.34041689-806d-49c6-98ff-51f452c7bf0b.meta' not found - ignoring {{(pid=129001) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}} Nov 27 21:14:05.771890 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Delete of backup '34041689-806d-49c6-98ff-51f452c7bf0b' for volume 'bc95af6a-3bbf-4795-81b0-cda182d7053e' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:14:05.780511 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:14:05.809408 ncdv-master-1 cinder-backup[129001]: WARNING cinder.quota [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Deprecated: Default quota for resource: backups is set by the default quota flag: quota_backups, it is now deprecated. Please use the default quota class for default quota. Nov 27 21:14:05.810313 ncdv-master-1 cinder-backup[129001]: WARNING cinder.quota [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Deprecated: Default quota for resource: backup_gigabytes is set by the default quota flag: quota_backup_gigabytes, it is now deprecated. Please use the default quota class for default quota. Nov 27 21:14:05.831752 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Created reservations ['54a2aef2-d544-4365-83f2-c39a70caf2e7', 'bd60268d-f739-475b-aa55-1026f27aa601'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:14:05.865812 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-ab6ecb83-9142-4c13-be5f-834ad76d6fd5 tempest-VolumesBackupsAdminTest-1080464940 None] Delete backup finished, backup 34041689-806d-49c6-98ff-51f452c7bf0b deleted. Nov 27 21:14:06.747054 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Delete backup started, backup: 3997df94-7bf1-415e-ae47-e84b32d51bb3. Nov 27 21:14:06.749624 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Delete started for backup=3997df94-7bf1-415e-ae47-e84b32d51bb3 {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:14:06.750247 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Trying diff format basename='volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' for backup base image of volume bc95af6a-3bbf-4795-81b0-cda182d7053e. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:14:06.803982 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Looking for snapshot of backup base 'volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:14:06.804405 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Found snapshot 'backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:14:06.804786 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Deleting backup snapshot='backup.3997df94-7bf1-415e-ae47-e84b32d51bb3.snap.1638047636.5048182' {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 27 21:14:07.420475 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Deleting backup base image='volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' of volume bc95af6a-3bbf-4795-81b0-cda182d7053e. Nov 27 21:14:07.679358 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Base backup image='volume-bc95af6a-3bbf-4795-81b0-cda182d7053e.backup.3997df94-7bf1-415e-ae47-e84b32d51bb3' of volume bc95af6a-3bbf-4795-81b0-cda182d7053e deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:14:07.732569 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Delete of backup '3997df94-7bf1-415e-ae47-e84b32d51bb3' for volume 'bc95af6a-3bbf-4795-81b0-cda182d7053e' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:14:07.739429 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:14:07.777346 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Created reservations ['284023c5-61f5-42ec-8752-9e59b859d7c8', '0002d994-dcdc-4e34-a529-bdd00ca1346a'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:14:07.824665 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-c90b984a-5097-445d-b29d-a7d7f0fc5c53 tempest-VolumesBackupsAdminTest-1903881971 None] Delete backup finished, backup 3997df94-7bf1-415e-ae47-e84b32d51bb3 deleted. Nov 27 21:14:10.316921 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Create backup started, backup: 6ae65962-f229-4a33-a7c9-110788191029 volume: e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8. Nov 27 21:14:10.332169 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:14:10.333603 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.345303 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:10.345643 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[0032bd1e-7a1c-4f40-b5c9-82a396f3e8cb]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:10.351152 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.359528 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ff130c30-7e04-4fbe-9ff6-ab7ac1a4c802]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:14:10.360053 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:14:10.361336 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff130c30-7e04-4fbe-9ff6-ab7ac1a4c802]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:10.363519 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.369115 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[87f9edd4-27f6-459e-b52a-24952b9fee86]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:14:10.369729 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:14:10.371318 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[87f9edd4-27f6-459e-b52a-24952b9fee86]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:10.371725 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.380636 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:10.380989 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[71b6ef01-4346-4b17-acf6-0e862dc93c6b]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:10.382808 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.391617 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:10.392091 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e09c91c-30b3-487b-8b6f-754efe8e1920]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:10.393298 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.399470 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:10.400165 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[72274a22-6b06-4e0c-a1ab-e52393ff06d6]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:14:10.401013 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:14:10.419593 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:14:10.420751 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] <== get_connector_properties: return (87ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:14:10.804866 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-77d2af62-f14e-4e15-b4e0-2fb0b29d193d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:14:11.102766 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:14:11.104189 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:14:11.105288 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:14:11.145561 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] <== connect_volume: return (40ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:14:11.163197 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Starting backup of volume='e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:14:11.164106 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Volume file is RBD: attempting optimized backup {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 27 21:14:11.183514 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Creating base image 'volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' {{(pid=129001) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 27 21:14:11.184018 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:14:11.221444 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Using --from-snap 'None' for incremental backup of volume e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8. {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 27 21:14:11.221852 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Creating backup snapshot='backup.6ae65962-f229-4a33-a7c9-110788191029.snap.1638047651.2214677' {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 27 21:14:11.972166 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Performing differential transfer from 'volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8' to 'volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:14:11.973235 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_gv61_3ew --pool volumes volumes/volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8@backup.6ae65962-f229-4a33-a7c9-110788191029.snap.1638047651.2214677 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:14:11.973611 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:14:13.012318 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Differential backup transfer completed in 1.0399s {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 27 21:14:13.013028 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:14:13.028276 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:14:13.028636 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:14:13.034919 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:14:13.035332 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:14:13.041395 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] No metadata type 'volume-glance-metadata' available {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 27 21:14:13.041939 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Backing up metadata for volume e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:14:13.073378 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Backup '6ae65962-f229-4a33-a7c9-110788191029' of volume e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8 finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:14:13.074929 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:14:13.092109 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] <== disconnect_volume: return (16ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:14:13.146774 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-00dfceca-0bb1-49db-9fd1-a5682ce2d007 tempest-VolumesBackupsAdminTest-1903881971 None] Create backup finished. backup: 6ae65962-f229-4a33-a7c9-110788191029. Nov 27 21:14:13.485872 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-c3567992-cca6-43d0-813a-aab4e29f603f tempest-VolumesBackupsAdminTest-1080464940 None] Reset backup status started, backup_id: 6ae65962-f229-4a33-a7c9-110788191029, status: error. Nov 27 21:14:13.486082 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-c3567992-cca6-43d0-813a-aab4e29f603f tempest-VolumesBackupsAdminTest-1080464940 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver. Nov 27 21:14:13.551481 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Delete backup started, backup: 6ae65962-f229-4a33-a7c9-110788191029. Nov 27 21:14:13.553939 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Delete started for backup=6ae65962-f229-4a33-a7c9-110788191029 {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:14:13.554365 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Trying diff format basename='volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' for backup base image of volume e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:14:13.599003 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Looking for snapshot of backup base 'volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:14:13.599439 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Found snapshot 'backup.6ae65962-f229-4a33-a7c9-110788191029.snap.1638047651.2214677' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:14:13.599823 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Deleting backup snapshot='backup.6ae65962-f229-4a33-a7c9-110788191029.snap.1638047651.2214677' {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 27 21:14:14.032918 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Deleting backup base image='volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' of volume e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8. Nov 27 21:14:14.200524 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Base backup image='volume-e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8.backup.6ae65962-f229-4a33-a7c9-110788191029' of volume e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8 deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:14:14.233821 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Delete of backup '6ae65962-f229-4a33-a7c9-110788191029' for volume 'e5a5170d-1ec9-45eb-9d9e-1ddf391b46d8' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:14:14.239477 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:14:14.263334 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Created reservations ['a3150050-f067-4dcb-9f6b-4865867dbef1', '8e73161c-83c4-4c67-a999-89849c221da0'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:14:14.287889 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-81e63d35-4a14-4216-b54c-b50a594895e4 tempest-VolumesBackupsAdminTest-1903881971 None] Delete backup finished, backup 6ae65962-f229-4a33-a7c9-110788191029 deleted. Nov 27 21:14:41.580572 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:14:41.581201 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:15:41.577416 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:15:41.578080 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:16:41.583123 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:16:41.583759 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:17:41.587602 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:17:41.588325 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:18:30.498104 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Create backup started, backup: 5df7eabd-c252-45f9-b580-4f8b0294e6e4 volume: 69f7cbd5-ca16-4ad9-b49d-dd66213e9539. Nov 27 21:18:30.506179 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:18:30.507610 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.514881 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:18:30.515213 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[99508c09-3f45-4152-ac02-31592076df67]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:18:30.516432 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.521663 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3c9f8f79-113e-46d7-9ac8-8c749f48b269]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:18:30.522310 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:18:30.523362 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c9f8f79-113e-46d7-9ac8-8c749f48b269]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:18:30.523769 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.529049 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[64ac6d50-3bdc-4d24-91b2-e0694aac0083]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:18:30.529601 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:18:30.530767 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[64ac6d50-3bdc-4d24-91b2-e0694aac0083]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:18:30.531215 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.538989 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:18:30.539341 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[a45137e2-94b6-4fd9-8543-a3142ae29707]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:18:30.540423 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.548966 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:18:30.549287 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[18a0fdeb-afc0-420d-8c0b-7c01fdf34f1c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:18:30.550336 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.556318 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:18:30.556649 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[30ddcbb1-3603-40bc-960b-810015aca146]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:18:30.558009 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:18:30.573962 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] CMD "nvme version" returned: 0 in 0.016s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:18:30.574977 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] <== get_connector_properties: return (67ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:18:30.847695 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-77d2af62-f14e-4e15-b4e0-2fb0b29d193d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:18:31.260247 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:18:31.261666 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '69f7cbd5-ca16-4ad9-b49d-dd66213e9539', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:18:31.262739 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:18:31.306214 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] <== connect_volume: return (44ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:18:31.324033 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Starting backup of volume='69f7cbd5-ca16-4ad9-b49d-dd66213e9539'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:18:31.324980 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Volume file is RBD: attempting optimized backup {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 27 21:18:31.345146 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Creating base image 'volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' {{(pid=129001) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 27 21:18:31.345637 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:18:31.377340 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Using --from-snap 'None' for incremental backup of volume 69f7cbd5-ca16-4ad9-b49d-dd66213e9539. {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 27 21:18:31.377746 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Creating backup snapshot='backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4.snap.1638047911.3773634' {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 27 21:18:32.207744 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Performing differential transfer from 'volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539' to 'volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:18:32.208432 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_2mdaq6lk --pool volumes volumes/volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539@backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4.snap.1638047911.3773634 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:18:32.208815 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:18:33.239175 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Differential backup transfer completed in 1.0307s {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 27 21:18:33.239804 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:18:33.253252 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:18:33.253671 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:18:33.260137 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:18:33.260549 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:18:33.266654 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] No metadata type 'volume-glance-metadata' available {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 27 21:18:33.267189 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Backing up metadata for volume 69f7cbd5-ca16-4ad9-b49d-dd66213e9539. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:18:33.298208 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Backup '5df7eabd-c252-45f9-b580-4f8b0294e6e4' of volume 69f7cbd5-ca16-4ad9-b49d-dd66213e9539 finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:18:33.299818 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '69f7cbd5-ca16-4ad9-b49d-dd66213e9539', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:18:33.322074 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] <== disconnect_volume: return (21ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:18:33.381133 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-f8b345a7-dfa0-4e89-8ffb-522a81f19ee1 tempest-VolumesBackupsV39Test-523004824 None] Create backup finished. backup: 5df7eabd-c252-45f9-b580-4f8b0294e6e4. Nov 27 21:18:33.703733 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Delete backup started, backup: 5df7eabd-c252-45f9-b580-4f8b0294e6e4. Nov 27 21:18:33.705596 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Delete started for backup=5df7eabd-c252-45f9-b580-4f8b0294e6e4 {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:18:33.706021 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Trying diff format basename='volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' for backup base image of volume 69f7cbd5-ca16-4ad9-b49d-dd66213e9539. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:18:33.755725 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Looking for snapshot of backup base 'volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:18:33.756183 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Found snapshot 'backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4.snap.1638047911.3773634' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:18:33.756563 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Deleting backup snapshot='backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4.snap.1638047911.3773634' {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 27 21:18:34.250473 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Deleting backup base image='volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' of volume 69f7cbd5-ca16-4ad9-b49d-dd66213e9539. Nov 27 21:18:34.419092 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Base backup image='volume-69f7cbd5-ca16-4ad9-b49d-dd66213e9539.backup.5df7eabd-c252-45f9-b580-4f8b0294e6e4' of volume 69f7cbd5-ca16-4ad9-b49d-dd66213e9539 deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:18:34.458132 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Delete of backup '5df7eabd-c252-45f9-b580-4f8b0294e6e4' for volume '69f7cbd5-ca16-4ad9-b49d-dd66213e9539' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:18:34.464264 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:18:34.486675 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Created reservations ['1e7105f0-a608-4c3c-ae18-e7afc1049041', 'ba683b1e-b508-4ea0-886a-15f8782cb772'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:18:34.511386 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-cc6da262-de44-4d2a-81ea-06cd4ca76e2b tempest-VolumesBackupsV39Test-523004824 None] Delete backup finished, backup 5df7eabd-c252-45f9-b580-4f8b0294e6e4 deleted. Nov 27 21:18:41.584843 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:18:41.585890 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:19:04.174673 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Create backup started, backup: 29ff5ee3-c571-494b-ad4d-769065dad63f volume: f4f886f9-5e7d-4b5a-bdf9-24c0605166d5 snapshot: 8ba87091-9c28-487a-a2a8-727c8531218a. Nov 27 21:19:04.182504 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:19:04.183993 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.191001 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:19:04.191369 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7ba0b2e-b8be-4973-bf8a-0f065e27233e]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:19:04.192570 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.198493 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3cacd091-7e57-415a-a8e7-d32b9fe310c5]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:19:04.199105 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:19:04.200243 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[3cacd091-7e57-415a-a8e7-d32b9fe310c5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:19:04.200714 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.206104 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[11c7cd17-f3a7-4c3d-894a-f41ee78d32c4]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:19:04.206673 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:19:04.207756 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[11c7cd17-f3a7-4c3d-894a-f41ee78d32c4]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:19:04.208338 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.216608 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:19:04.216921 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[637b5d08-5b7e-4911-af65-6e95dbbbfd88]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:19:04.218374 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.226231 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:19:04.226544 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[f93327a0-19c0-4a65-8053-490ab75be3de]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:19:04.227691 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.233920 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:19:04.234213 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[75fd0808-bbdb-4e25-89dd-498904a9ce41]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:19:04.235083 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:19:04.251569 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] CMD "nvme version" returned: 0 in 0.016s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:19:04.252680 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] <== get_connector_properties: return (69ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:19:05.144593 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:19:05.146169 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-a898c726-eed0-40f8-b97e-53b5e8bac437', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a898c726-eed0-40f8-b97e-53b5e8bac437', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:19:05.147273 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:19:05.202601 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] <== connect_volume: return (55ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:19:05.210005 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:19:05.228218 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Starting backup of volume='f4f886f9-5e7d-4b5a-bdf9-24c0605166d5'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:19:05.248124 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:19:05.248527 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Creating backup base image='volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.base' for volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5. {{(pid=129001) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}} Nov 27 21:19:05.275991 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Copying data from volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5. {{(pid=129001) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}} Nov 27 21:19:05.293325 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferring data between 'volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5' and 'volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.base' {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}} Nov 27 21:19:05.293748 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] 8 chunks of 134217728 bytes to be transferred {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}} Nov 27 21:19:06.240475 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 1 of 8 (138541K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:07.175777 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 2 of 8 (140238K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:08.139860 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 3 of 8 (136062K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:09.110759 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 4 of 8 (135072K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:10.090986 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 5 of 8 (133802K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:11.066491 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 6 of 8 (134452K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:12.002714 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 7 of 8 (140096K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:12.968204 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Transferred chunk 8 of 8 (135841K/s) {{(pid=129001) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 27 21:19:13.010384 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:19:13.020768 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:19:13.021179 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:19:13.027180 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:19:13.027588 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:19:13.033548 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] No metadata type 'volume-glance-metadata' available {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 27 21:19:13.034083 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Backing up metadata for volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:19:13.064005 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Backup '29ff5ee3-c571-494b-ad4d-769065dad63f' of volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5 finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:19:13.065331 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-a898c726-eed0-40f8-b97e-53b5e8bac437', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a898c726-eed0-40f8-b97e-53b5e8bac437', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:19:13.078544 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] <== disconnect_volume: return (12ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:19:13.154197 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-7fe07f7d-7332-4035-93bb-c399ff7e5c35 tempest-VolumesSnapshotTestJSON-481511107 None] Create backup finished. backup: 29ff5ee3-c571-494b-ad4d-769065dad63f. Nov 27 21:19:13.477586 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Delete backup started, backup: 29ff5ee3-c571-494b-ad4d-769065dad63f. Nov 27 21:19:13.479099 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Delete started for backup=29ff5ee3-c571-494b-ad4d-769065dad63f {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:19:13.479554 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Trying diff format basename='volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.29ff5ee3-c571-494b-ad4d-769065dad63f' for backup base image of volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:19:13.508957 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Image 'volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.29ff5ee3-c571-494b-ad4d-769065dad63f' not found - trying diff format name {{(pid=129001) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}} Nov 27 21:19:13.527594 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Looking for snapshot of backup base 'volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.base' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:19:13.527971 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Backup base 'volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.base' has no snapshots {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}} Nov 27 21:19:13.528338 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] No backup snapshot to delete {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Nov 27 21:19:13.534169 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Deleting backup base image='volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.base' of volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5. Nov 27 21:19:14.005413 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Base backup image='volume-f4f886f9-5e7d-4b5a-bdf9-24c0605166d5.backup.base' of volume f4f886f9-5e7d-4b5a-bdf9-24c0605166d5 deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:19:14.041014 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Delete of backup '29ff5ee3-c571-494b-ad4d-769065dad63f' for volume 'f4f886f9-5e7d-4b5a-bdf9-24c0605166d5' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:19:14.046523 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:19:14.069955 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Created reservations ['52200488-1b83-4ae0-8762-77911e6cc8a2', '81c71f87-e3d0-4b8d-84f5-b74ec51140fc'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:19:14.093508 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-439023f0-9339-403b-8541-5940170890f3 tempest-VolumesSnapshotTestJSON-481511107 None] Delete backup finished, backup 29ff5ee3-c571-494b-ad4d-769065dad63f deleted. Nov 27 21:19:41.581244 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:19:41.581906 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:20:41.586833 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:20:41.587494 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:21:41.587969 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:21:41.588579 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:22:41.593531 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:22:41.594146 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:23:12.344832 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Create backup started, backup: 8bb68315-6043-4cce-a567-23794fbb2f85 volume: fdd54218-1833-48c7-b2c5-a71771d4a096. Nov 27 21:23:12.352768 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:12.354063 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.361683 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:12.362005 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[2766e231-4972-40e5-ac09-0d031df46466]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:12.363334 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.369009 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[854cb7ea-c51e-4291-8357-e16ebb4e6b22]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:23:12.369616 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:23:12.370742 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[854cb7ea-c51e-4291-8357-e16ebb4e6b22]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:12.371016 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.376894 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[02c1aca2-c255-4706-8098-64780b7aae5c]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:23:12.377469 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:23:12.378637 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[02c1aca2-c255-4706-8098-64780b7aae5c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:12.379130 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.387115 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:12.387441 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[aae943f4-1cad-4574-b416-268b9fb0d4d6]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:12.388480 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.396969 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:12.397293 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a515086-f68e-4c62-b489-65dc50bff83a]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:12.398726 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.404839 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:12.405331 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d8e7a1e-fb6d-4155-addf-967e26ab5713]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:12.406035 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:12.424466 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:12.425545 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] <== get_connector_properties: return (72ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:13.126442 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:23:13.127856 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-fdd54218-1833-48c7-b2c5-a71771d4a096', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'fdd54218-1833-48c7-b2c5-a71771d4a096', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:13.128923 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:23:13.166824 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] <== connect_volume: return (38ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:13.174449 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:23:13.191380 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Starting backup of volume='fdd54218-1833-48c7-b2c5-a71771d4a096'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:23:13.192541 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Volume file is RBD: attempting optimized backup {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 27 21:23:13.212176 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Creating base image 'volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' {{(pid=129001) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 27 21:23:13.212584 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:23:13.242904 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Using --from-snap 'None' for incremental backup of volume fdd54218-1833-48c7-b2c5-a71771d4a096. {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 27 21:23:13.243341 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Creating backup snapshot='backup.8bb68315-6043-4cce-a567-23794fbb2f85.snap.1638048193.2429283' {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 27 21:23:13.671950 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Performing differential transfer from 'volume-fdd54218-1833-48c7-b2c5-a71771d4a096' to 'volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:23:13.672571 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_zjr8i8xg --pool volumes volumes/volume-fdd54218-1833-48c7-b2c5-a71771d4a096@backup.8bb68315-6043-4cce-a567-23794fbb2f85.snap.1638048193.2429283 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:23:13.672663 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:23:14.719632 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Differential backup transfer completed in 1.0473s {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 27 21:23:14.720881 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:23:14.734177 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:23:14.734602 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:23:14.741255 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:23:14.741640 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:23:14.748307 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] No metadata type 'volume-glance-metadata' available {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 27 21:23:14.748449 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Backing up metadata for volume fdd54218-1833-48c7-b2c5-a71771d4a096. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:23:14.790457 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Backup '8bb68315-6043-4cce-a567-23794fbb2f85' of volume fdd54218-1833-48c7-b2c5-a71771d4a096 finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:23:14.792043 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-fdd54218-1833-48c7-b2c5-a71771d4a096', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'fdd54218-1833-48c7-b2c5-a71771d4a096', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:14.822638 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] <== disconnect_volume: return (29ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:14.893100 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-8fbeede7-a9be-49c3-8f89-7699295c7800 tempest-VolumesBackupsTest-1517995977 None] Create backup finished. backup: 8bb68315-6043-4cce-a567-23794fbb2f85. Nov 27 21:23:15.504961 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Delete backup started, backup: 8bb68315-6043-4cce-a567-23794fbb2f85. Nov 27 21:23:15.506977 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Delete started for backup=8bb68315-6043-4cce-a567-23794fbb2f85 {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:23:15.507441 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Trying diff format basename='volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' for backup base image of volume fdd54218-1833-48c7-b2c5-a71771d4a096. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:23:15.559438 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Looking for snapshot of backup base 'volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:23:15.559869 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Found snapshot 'backup.8bb68315-6043-4cce-a567-23794fbb2f85.snap.1638048193.2429283' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:23:15.560276 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Deleting backup snapshot='backup.8bb68315-6043-4cce-a567-23794fbb2f85.snap.1638048193.2429283' {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 27 21:23:16.492054 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Deleting backup base image='volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' of volume fdd54218-1833-48c7-b2c5-a71771d4a096. Nov 27 21:23:16.670017 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Base backup image='volume-fdd54218-1833-48c7-b2c5-a71771d4a096.backup.8bb68315-6043-4cce-a567-23794fbb2f85' of volume fdd54218-1833-48c7-b2c5-a71771d4a096 deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:23:16.705436 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Delete of backup '8bb68315-6043-4cce-a567-23794fbb2f85' for volume 'fdd54218-1833-48c7-b2c5-a71771d4a096' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:23:16.711000 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:23:16.732608 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Created reservations ['7f72d227-328d-40c8-914d-a9a27bcdb9cf', '394b6716-44b9-44bf-83cf-cf956ff22e14'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:23:16.759193 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-40c7c477-0470-4229-a0cf-c207c6e7a4c2 tempest-VolumesBackupsTest-1517995977 None] Delete backup finished, backup 8bb68315-6043-4cce-a567-23794fbb2f85 deleted. Nov 27 21:23:27.428990 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Create backup started, backup: 0d4c34ac-d139-49d9-9e6b-b4258ea64006 volume: 70893a5c-b091-4700-bd50-978ce73dc56b. Nov 27 21:23:27.436876 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:27.438282 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.449803 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:27.450186 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[41bdbc2d-3878-44d3-9dd5-5ec5de5bbe23]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:27.451610 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.457012 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[9d510ff0-3ed6-4a79-915e-5ea115d09284]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:23:27.457859 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:23:27.459082 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d510ff0-3ed6-4a79-915e-5ea115d09284]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:27.459431 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.464360 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b2ebbe6a-655d-4757-b12c-4e426f956550]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:23:27.464913 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:23:27.466094 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2ebbe6a-655d-4757-b12c-4e426f956550]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:27.466845 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.474721 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:27.475072 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[d382636f-8d65-4e67-8d71-fcb9e77da227]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:27.476160 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.484028 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:27.484382 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3d1aedd-81df-4739-aab2-e807b2f2d42a]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:27.485894 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.493801 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:27.494341 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[629f140d-992e-4e7c-9388-41f2374bdbbd]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:27.495067 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:27.513427 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:27.514490 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] <== get_connector_properties: return (77ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:28.196766 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:23:28.198208 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-70893a5c-b091-4700-bd50-978ce73dc56b', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '70893a5c-b091-4700-bd50-978ce73dc56b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:28.199278 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:23:28.260122 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] <== connect_volume: return (61ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:28.268036 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:23:28.285381 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Starting backup of volume='70893a5c-b091-4700-bd50-978ce73dc56b'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:23:28.286340 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Volume file is RBD: attempting optimized backup {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 27 21:23:28.305476 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Creating base image 'volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' {{(pid=129001) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 27 21:23:28.305881 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:23:28.341347 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Using --from-snap 'None' for incremental backup of volume 70893a5c-b091-4700-bd50-978ce73dc56b. {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 27 21:23:28.341752 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Creating backup snapshot='backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694' {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 27 21:23:28.634022 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Performing differential transfer from 'volume-70893a5c-b091-4700-bd50-978ce73dc56b' to 'volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:23:28.634713 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_s90ba02i --pool volumes volumes/volume-70893a5c-b091-4700-bd50-978ce73dc56b@backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:23:28.635033 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:23:34.605029 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Differential backup transfer completed in 5.9708s {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 27 21:23:34.605653 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:23:34.617687 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:23:34.618115 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:23:34.624828 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:23:34.625218 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:23:34.634061 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 27 21:23:34.634544 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Backing up metadata for volume 70893a5c-b091-4700-bd50-978ce73dc56b. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:23:34.671634 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Backup '0d4c34ac-d139-49d9-9e6b-b4258ea64006' of volume 70893a5c-b091-4700-bd50-978ce73dc56b finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:23:34.673435 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-70893a5c-b091-4700-bd50-978ce73dc56b', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '70893a5c-b091-4700-bd50-978ce73dc56b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:34.694218 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] <== disconnect_volume: return (19ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:34.787973 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-3d251613-9d0a-492b-9d5c-b6c11bc8b940 tempest-VolumesBackupsTest-1517995977 None] Create backup finished. backup: 0d4c34ac-d139-49d9-9e6b-b4258ea64006. Nov 27 21:23:36.939657 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Restore backup started, backup: 0d4c34ac-d139-49d9-9e6b-b4258ea64006 volume: 96161016-d59e-4579-b4c2-6215c2dc63be. Nov 27 21:23:36.952924 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:36.954371 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:36.966740 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:36.967097 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[d34aec8f-cd72-4789-97e6-827251d61d31]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:36.968361 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:36.974373 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b82a4881-6d00-47c5-bbc1-f2ac1b9e908c]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:23:36.974969 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:23:36.976163 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[b82a4881-6d00-47c5-bbc1-f2ac1b9e908c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:36.976380 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:36.981598 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c01e06b2-90b2-40c9-b99c-2820893fa250]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:23:36.982086 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:23:36.983305 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[c01e06b2-90b2-40c9-b99c-2820893fa250]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:36.983809 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:36.992570 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:36.992881 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd55b150-c05e-49c3-83b3-003f9a1299db]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:36.993990 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:37.001880 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:37.002201 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[73387b51-31d8-4f33-961c-0885b32fbc26]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:37.003324 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:37.009340 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:37.009725 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[1283dfd5-a496-4014-97d6-96ccaf838292]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:23:37.010518 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:23:37.029346 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:23:37.030388 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] <== get_connector_properties: return (77ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:37.813298 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:23:37.815514 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-96161016-d59e-4579-b4c2-6215c2dc63be', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '96161016-d59e-4579-b4c2-6215c2dc63be', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:37.816673 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:23:37.861984 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] <== connect_volume: return (45ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:37.869166 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:23:37.881117 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Starting restore from Ceph backup=0d4c34ac-d139-49d9-9e6b-b4258ea64006 to volume=volume-96161016-d59e-4579-b4c2-6215c2dc63be {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 27 21:23:37.953753 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Looking for snapshot of backup base 'volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:23:37.954555 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Found snapshot 'backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:23:37.960402 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Volume file is RBD. {{(pid=129001) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 27 21:23:38.010655 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Attempting differential restore. {{(pid=129001) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 27 21:23:38.011233 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Attempting incremental restore from base='volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' snap='backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694' {{(pid=129001) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 27 21:23:38.011716 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Performing differential transfer from 'volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' to 'volume-96161016-d59e-4579-b4c2-6215c2dc63be' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:23:38.012215 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006@backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:23:38.012653 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_j16mrd5o --pool volumes - volumes/volume-96161016-d59e-4579-b4c2-6215c2dc63be' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:23:41.598971 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:23:41.599564 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:23:41.772190 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Restore transfer completed in 3.7603s {{(pid=129001) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 27 21:23:41.773262 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 27 21:23:41.795056 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Restoring volume base metadata {{(pid=129001) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 27 21:23:41.813050 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] No metadata of type 'volume-metadata' to restore {{(pid=129001) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 27 21:23:41.813630 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Restoring volume glance metadata {{(pid=129001) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 27 21:23:41.946354 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Restore to volume 96161016-d59e-4579-b4c2-6215c2dc63be finished successfully. {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 27 21:23:41.947943 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-96161016-d59e-4579-b4c2-6215c2dc63be', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '96161016-d59e-4579-b4c2-6215c2dc63be', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:23:41.957653 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] <== disconnect_volume: return (9ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:23:42.005935 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.manager [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Encryption key ID for volume 96161016-d59e-4579-b4c2-6215c2dc63be already matches encryption key ID in backup 0d4c34ac-d139-49d9-9e6b-b4258ea64006. {{(pid=129001) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 27 21:23:42.034526 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-c4dc1d7a-c788-4855-9db4-450c6995fe2b tempest-VolumesBackupsTest-1517995977 None] Finished restoring backup 0d4c34ac-d139-49d9-9e6b-b4258ea64006 to volume 96161016-d59e-4579-b4c2-6215c2dc63be. Nov 27 21:23:43.380715 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Delete backup started, backup: 0d4c34ac-d139-49d9-9e6b-b4258ea64006. Nov 27 21:23:43.382549 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Delete started for backup=0d4c34ac-d139-49d9-9e6b-b4258ea64006 {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:23:43.383001 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Trying diff format basename='volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' for backup base image of volume 70893a5c-b091-4700-bd50-978ce73dc56b. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:23:43.432895 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Looking for snapshot of backup base 'volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:23:43.433286 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Found snapshot 'backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:23:43.433664 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Deleting backup snapshot='backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006.snap.1638048208.3413694' {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 27 21:23:43.742867 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Deleting backup base image='volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' of volume 70893a5c-b091-4700-bd50-978ce73dc56b. Nov 27 21:23:44.100684 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Base backup image='volume-70893a5c-b091-4700-bd50-978ce73dc56b.backup.0d4c34ac-d139-49d9-9e6b-b4258ea64006' of volume 70893a5c-b091-4700-bd50-978ce73dc56b deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:23:44.134960 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Delete of backup '0d4c34ac-d139-49d9-9e6b-b4258ea64006' for volume '70893a5c-b091-4700-bd50-978ce73dc56b' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:23:44.140950 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:23:44.163794 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Created reservations ['5fa8ffdb-e832-409d-a1a1-a8ae5d82124d', 'e1caf3a8-11f7-4bff-8bcc-7b963040895d'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:23:44.187684 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-66a18993-cd5e-4031-a458-035ca3b2aac6 tempest-VolumesBackupsTest-1517995977 None] Delete backup finished, backup 0d4c34ac-d139-49d9-9e6b-b4258ea64006 deleted. Nov 27 21:24:41.595526 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:24:41.596128 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:25:41.591200 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:25:41.591925 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:26:41.596506 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:26:41.597247 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:27:41.602140 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:27:41.602857 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:27:55.125216 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Create backup started, backup: 6906a280-43f6-416c-9528-e316f4a00ceb volume: 895f0bf1-7373-4777-9163-1d0dff41ae31. Nov 27 21:27:55.133654 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:27:55.135187 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.144232 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:27:55.144733 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4cc86db-0769-4532-ac47-cc2fd82b8c2a]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:27:55.147691 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.152757 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[1708e47a-82af-4d06-9e55-67358536d78b]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:27:55.153306 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:27:55.154422 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[1708e47a-82af-4d06-9e55-67358536d78b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:27:55.154785 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.160503 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[96f9908e-838e-4e21-812d-f38e6b01cf73]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:27:55.161069 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:27:55.162389 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[96f9908e-838e-4e21-812d-f38e6b01cf73]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:27:55.163378 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.172346 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:27:55.172678 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[fecd005f-a24c-4321-a6d1-28cd0479c9dc]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:27:55.173804 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.182331 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:27:55.182649 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[3035523c-f909-4135-bf98-3f38b4435dbd]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:27:55.184360 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.190804 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:27:55.191137 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ae97e5b-9ea3-4450-abbe-a847eddafc15]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:27:55.191896 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:27:55.210999 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:27:55.212170 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] <== get_connector_properties: return (77ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:27:55.925382 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:27:55.926798 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-895f0bf1-7373-4777-9163-1d0dff41ae31', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '895f0bf1-7373-4777-9163-1d0dff41ae31', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:27:55.927968 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:27:55.993986 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] <== connect_volume: return (66ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:27:56.002298 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:27:56.020167 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Starting backup of volume='895f0bf1-7373-4777-9163-1d0dff41ae31'. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 27 21:27:56.021283 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Volume file is RBD: attempting optimized backup {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 27 21:27:56.040507 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Creating base image 'volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' {{(pid=129001) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 27 21:27:56.040923 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129001) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 27 21:27:56.072512 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Using --from-snap 'None' for incremental backup of volume 895f0bf1-7373-4777-9163-1d0dff41ae31. {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 27 21:27:56.072927 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Creating backup snapshot='backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538' {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 27 21:27:56.749773 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Performing differential transfer from 'volume-895f0bf1-7373-4777-9163-1d0dff41ae31' to 'volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:27:56.750444 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_p_6thkqj --pool volumes volumes/volume-895f0bf1-7373-4777-9163-1d0dff41ae31@backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:27:56.750558 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:27:59.822402 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Differential backup transfer completed in 3.0724s {{(pid=129001) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 27 21:27:59.823093 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Getting metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 27 21:27:59.836695 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129001) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 27 21:27:59.837110 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Getting metadata type 'volume-metadata' {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 27 21:27:59.843936 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] No metadata type 'volume-metadata' available {{(pid=129001) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 27 21:27:59.844316 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Getting metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 27 21:27:59.851165 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=129001) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 27 21:27:59.851762 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Backing up metadata for volume 895f0bf1-7373-4777-9163-1d0dff41ae31. {{(pid=129001) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 27 21:27:59.882770 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Backup '6906a280-43f6-416c-9528-e316f4a00ceb' of volume 895f0bf1-7373-4777-9163-1d0dff41ae31 finished. {{(pid=129001) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 27 21:27:59.884396 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-895f0bf1-7373-4777-9163-1d0dff41ae31', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '895f0bf1-7373-4777-9163-1d0dff41ae31', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:27:59.903839 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] <== disconnect_volume: return (18ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:27:59.958261 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-af5b2f7f-f8db-421b-9296-6c044b7f2955 tempest-TestVolumeBackupRestore-900720088 None] Create backup finished. backup: 6906a280-43f6-416c-9528-e316f4a00ceb. Nov 27 21:28:01.566049 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Restore backup started, backup: 6906a280-43f6-416c-9528-e316f4a00ceb volume: b896c246-d90f-40d7-8c0c-b3d5febff035. Nov 27 21:28:01.579499 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.21', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:28:01.580850 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.588541 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:28:01.588883 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[43e6ebcd-ad89-4956-99f1-fbc1d19f64d4]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ebf1d01dde5c\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:28:01.590714 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.597187 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3c336263-fe02-426a-b679-2ac1b610e57c]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:28:01.597683 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:28:01.598959 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c336263-fe02-426a-b679-2ac1b610e57c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:28:01.599212 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.604774 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8a5da296-385c-4a71-a85f-85ea73e74954]: [Errno 2] No such file or directory: 'systool' {{(pid=165020) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: Traceback (most recent call last): Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: ret = func(*f_args, **f_kwargs) Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: return func(*args, **kwargs) Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: return putils.execute(on_execute=on_execute, Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: obj = subprocess.Popen(cmd, Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 27 21:28:01.605975 ncdv-master-1 cinder-backup[129001]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 27 21:28:01.607234 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a5da296-385c-4a71-a85f-85ea73e74954]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:28:01.608142 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.626788 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:28:01.627114 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb35ac74-d324-4ffc-955a-3a0bd36f0b23]: (4, ('/dev/sda1\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:28:01.628192 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.638631 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:28:01.638948 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbfd8beb-df2d-45af-a18a-70128ffada21]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:28:01.640011 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.646538 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=165020) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:28:01.646868 ncdv-master-1 cinder-backup[129001]: DEBUG oslo.privsep.daemon [-] privsep: reply[c02dffe9-5e7d-4d72-bea7-ee7a7dd287c6]: (4, ('af49b4f0-8519-c04c-b40b-aab7a0a5efc5\n', '')) {{(pid=165020) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 27 21:28:01.647750 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Running cmd (subprocess): nvme version {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 27 21:28:01.673106 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_concurrency.processutils [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] CMD "nvme version" returned: 0 in 0.025s {{(pid=129001) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 27 21:28:01.674193 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.utils [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] <== get_connector_properties: return (94ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.21', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ebf1d01dde5c', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'af49b4f0-8519-c04c-b40b-aab7a0a5efc5', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:bc037a1f-107b-4c5d-97f0-c589fd4ca879'} {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:28:02.336020 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connector [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Factory for rbd on None {{(pid=129001) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 27 21:28:02.337515 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-b896c246-d90f-40d7-8c0c-b3d5febff035', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b896c246-d90f-40d7-8c0c-b3d5febff035', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:28:02.338610 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.linuxrbd [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] opening connection to ceph cluster (timeout=-1). {{(pid=129001) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 27 21:28:02.379172 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] <== connect_volume: return (41ms) {'path': } {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:28:02.386280 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:28:02.397421 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Starting restore from Ceph backup=6906a280-43f6-416c-9528-e316f4a00ceb to volume=volume-b896c246-d90f-40d7-8c0c-b3d5febff035 {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 27 21:28:02.454118 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Looking for snapshot of backup base 'volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:28:02.454522 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Found snapshot 'backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:28:02.458673 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Volume file is RBD. {{(pid=129001) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 27 21:28:02.491045 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Attempting differential restore. {{(pid=129001) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 27 21:28:02.491575 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Attempting incremental restore from base='volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' snap='backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538' {{(pid=129001) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 27 21:28:02.491973 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Performing differential transfer from 'volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' to 'volume-b896c246-d90f-40d7-8c0c-b3d5febff035' {{(pid=129001) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 27 21:28:02.492635 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb@backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538 -' into... {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 27 21:28:02.492993 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_qaqfe9eg --pool volumes - volumes/volume-b896c246-d90f-40d7-8c0c-b3d5febff035' {{(pid=129001) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 27 21:28:06.663674 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Restore transfer completed in 4.1711s {{(pid=129001) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 27 21:28:06.664668 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 27 21:28:06.685882 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Restoring volume base metadata {{(pid=129001) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 27 21:28:06.705848 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] No metadata of type 'volume-metadata' to restore {{(pid=129001) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 27 21:28:06.706188 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.driver [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Restoring volume glance metadata {{(pid=129001) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 27 21:28:06.839862 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Restore to volume b896c246-d90f-40d7-8c0c-b3d5febff035 finished successfully. {{(pid=129001) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 27 21:28:06.841948 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-b896c246-d90f-40d7-8c0c-b3d5febff035', 'hosts': ['10.222.0.21'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b896c246-d90f-40d7-8c0c-b3d5febff035', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 27 21:28:06.854133 ncdv-master-1 cinder-backup[129001]: DEBUG os_brick.initiator.connectors.rbd [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] <== disconnect_volume: return (11ms) None {{(pid=129001) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 27 21:28:06.900134 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.manager [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Encryption key ID for volume b896c246-d90f-40d7-8c0c-b3d5febff035 already matches encryption key ID in backup 6906a280-43f6-416c-9528-e316f4a00ceb. {{(pid=129001) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 27 21:28:06.929033 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-5136a9bc-b4f5-4896-8fe5-e06fcf0ee72b tempest-TestVolumeBackupRestore-900720088 None] Finished restoring backup 6906a280-43f6-416c-9528-e316f4a00ceb to volume b896c246-d90f-40d7-8c0c-b3d5febff035. Nov 27 21:28:41.599291 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:28:41.599966 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:29:10.047587 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Delete backup started, backup: 6906a280-43f6-416c-9528-e316f4a00ceb. Nov 27 21:29:10.049222 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Delete started for backup=6906a280-43f6-416c-9528-e316f4a00ceb {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 27 21:29:10.049663 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Trying diff format basename='volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' for backup base image of volume 895f0bf1-7373-4777-9163-1d0dff41ae31. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 27 21:29:10.115482 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Looking for snapshot of backup base 'volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 27 21:29:10.116129 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Found snapshot 'backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538' {{(pid=129001) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 27 21:29:10.116589 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Deleting backup snapshot='backup.6906a280-43f6-416c-9528-e316f4a00ceb.snap.1638048476.072538' {{(pid=129001) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 27 21:29:10.635048 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Deleting backup base image='volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' of volume 895f0bf1-7373-4777-9163-1d0dff41ae31. Nov 27 21:29:10.983021 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_db.sqlalchemy.engines [None req-77d2af62-f14e-4e15-b4e0-2fb0b29d193d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129001) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 27 21:29:11.141544 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Base backup image='volume-895f0bf1-7373-4777-9163-1d0dff41ae31.backup.6906a280-43f6-416c-9528-e316f4a00ceb' of volume 895f0bf1-7373-4777-9163-1d0dff41ae31 deleted. {{(pid=129001) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 27 21:29:11.178892 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.backup.drivers.ceph [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Delete of backup '6906a280-43f6-416c-9528-e316f4a00ceb' for volume '895f0bf1-7373-4777-9163-1d0dff41ae31' finished. {{(pid=129001) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 27 21:29:11.202327 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.quota [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Created reservations ['027d3663-05ad-45b5-bc13-334c307714f2', '898a7516-5bcf-4adf-b2a0-d690f675a21e'] {{(pid=129001) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 27 21:29:11.228045 ncdv-master-1 cinder-backup[129001]: INFO cinder.backup.manager [None req-092ebab4-45bb-4681-ae28-e1b529ef93b1 tempest-TestVolumeBackupRestore-900720088 None] Delete backup finished, backup 6906a280-43f6-416c-9528-e316f4a00ceb deleted. Nov 27 21:29:41.604700 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:29:41.605347 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:31:41.610290 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:31:41.611020 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:32:41.615451 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:32:41.616130 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:33:41.611337 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:33:41.611936 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:34:41.616454 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:34:41.617219 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:35:41.613785 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:35:41.614380 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:36:41.618522 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:36:41.619153 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:37:41.623680 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:37:41.624314 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:38:41.628970 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:38:41.629531 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:39:41.634120 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:39:41.634722 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:40:41.639875 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:40:41.640482 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:41:41.644534 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:41:41.645285 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:42:41.650267 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:42:41.650848 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:43:41.646994 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:43:41.647718 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:44:41.652687 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:44:41.653389 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:45:41.658167 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:45:41.658913 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:46:41.660001 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:46:41.660695 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:47:41.664586 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:47:41.665245 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:48:41.660419 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:48:41.661167 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:49:41.665061 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:49:41.665744 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:50:41.669890 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:50:41.671766 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:51:41.667296 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:51:41.668065 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:52:41.672809 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:52:41.673481 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:53:41.681420 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:53:41.687900 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:54:41.683435 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:54:41.684124 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:55:41.689742 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:55:41.691006 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:56:41.685840 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:56:41.686588 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:57:41.681707 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:57:41.682868 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:58:41.681667 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:58:41.682255 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 21:59:41.687005 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 21:59:41.687776 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:00:41.692311 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:00:41.693066 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:01:41.698013 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:01:41.698748 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:02:41.694873 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:02:41.696120 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:03:41.692008 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:03:41.692720 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:04:41.697352 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:04:41.698002 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:05:41.702121 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:05:41.702677 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:06:41.707515 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:06:41.708140 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:07:41.712988 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:07:41.713631 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:08:41.709303 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:08:41.709968 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:09:41.714475 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:09:41.715097 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:10:41.719265 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:10:41.719877 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:11:41.723934 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:11:41.724538 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:12:41.729170 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:12:41.729813 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:13:41.734068 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:13:41.734688 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:14:41.739847 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:14:41.740471 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:15:41.744751 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:15:41.745379 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:16:41.749477 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:16:41.750122 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:17:41.745008 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:17:41.745732 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:18:41.745239 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:18:41.745922 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:19:41.750166 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:19:41.750887 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:20:41.746952 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:20:41.747617 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:21:41.744429 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:21:41.745087 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:22:41.740437 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:22:41.741085 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:23:41.745373 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:23:41.746008 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:24:41.750499 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:24:41.751157 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:25:41.755066 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:25:41.755764 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:26:41.751902 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:26:41.752582 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 27 22:27:41.756520 ncdv-master-1 cinder-backup[129001]: DEBUG oslo_service.periodic_task [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129001) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 27 22:27:41.757199 ncdv-master-1 cinder-backup[129001]: DEBUG cinder.manager [None req-af0a11bd-3a54-47da-8d0b-721c38608634 None None] Notifying Schedulers of capabilities ... {{(pid=129001) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}