-- Logs begin at Sat 2021-11-13 20:01:14 UTC, end at Sat 2021-11-13 22:42:33 UTC. -- Nov 13 20:55:07.260992 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service. Nov 13 20:55:10.660046 ncdv-master-1 cinder-backup[128544]: INFO cinder.cmd.backup [-] Backup running with 1 processes. Nov 13 20:55:10.661013 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=128544) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Nov 13 20:55:10.661187 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=128544) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Nov 13 20:55:10.836931 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_db.sqlalchemy.engines [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=128544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 20:55:11.062537 ncdv-master-1 cinder-backup[128544]: INFO cinder.rpc [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Nov 13 20:55:11.066592 ncdv-master-1 cinder-backup[128544]: INFO cinder.rpc [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version. Nov 13 20:55:11.070833 ncdv-master-1 cinder-backup[128544]: INFO cinder.rpc [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Nov 13 20:55:11.074507 ncdv-master-1 cinder-backup[128544]: INFO cinder.rpc [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version. Nov 13 20:55:11.078896 ncdv-master-1 cinder-backup[128544]: INFO cinder.rpc [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Nov 13 20:55:11.083379 ncdv-master-1 cinder-backup[128544]: INFO cinder.rpc [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Nov 13 20:55:11.134393 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_db.sqlalchemy.engines [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=128544) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 20:55:11.262738 ncdv-master-1 cinder-backup[128544]: INFO oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Starting 1 workers Nov 13 20:55:11.268356 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Started child 129022 {{(pid=128544) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Nov 13 20:55:11.276173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Full set of CONF: {{(pid=128544) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Nov 13 20:55:11.276173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] ******************************************************************************** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 13 20:55:11.276173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] Configuration options gathered from: {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 13 20:55:11.276173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 13 20:55:11.276173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 13 20:55:11.276173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] ================================================================================ {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 13 20:55:11.276673 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] allow_availability_zone_fallback = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.276673 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] allow_compression_on_image_upload = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.276673 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] allowed_direct_url_schemes = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.276673 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] api_rate_limit = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.276673 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] auth_strategy = keystone {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.276673 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_availability_zone = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.278159 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_api_class = cinder.backup.api.API {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.278662 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_chunk_size = 134217728 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.279090 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_conf = /etc/ceph/ceph.conf {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.279449 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_image_journals = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.282469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_pool = backups {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.284138 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_stripe_count = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.286140 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_stripe_unit = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.286569 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_ceph_user = cinder-bak {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.287079 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_driver = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.287474 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_driver_init_check_interval = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.287901 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_driver_stats_polling_interval = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.290257 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.291869 ncdv-master-1 cinder-backup[129022]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0) Nov 13 20:55:11.292369 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_max_operations = 15 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.292447 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_metadata_version = 2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.293725 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_name_template = backup-%s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.294099 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_native_threads_pool_size = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.295099 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_object_number_per_notification = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.295470 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_service_inithost_offload = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.295833 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_timer_interval = 120 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.296230 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_use_same_host = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.300131 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_use_temp_snapshot = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.300562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backup_workers = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.301493 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] chap_password = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.302867 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] chap_username = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.303567 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.304008 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_internal_tenant_project_id = 5c62bfce2e1c42ff851da53cfbfff43b {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.304418 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_internal_tenant_user_id = 2ea30c0bd53c4192bb4cd197aa27e8d2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.304842 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] compression_format = gzip {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.309634 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] config_dir = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.309959 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.310708 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] config_source = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.311518 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.312339 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] control_exchange = openstack {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.312848 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] debug = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.316665 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] default_availability_zone = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.317184 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] default_group_type = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.317800 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.320662 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] default_volume_type = ceph {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.322014 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] driver_client_cert = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.324877 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] driver_client_cert_key = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.325360 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] driver_data_namespace = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] driver_ssl_cert_path = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] driver_ssl_cert_verify = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] driver_use_ssl = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] enable_new_services = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] enable_unsupported_driver = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] enabled_backends = ['ceph'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.344895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] enforce_multipath_for_image_xfer = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345402 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] filter_function = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345402 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_api_insecure = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345402 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_api_servers = ['http://10.222.0.20/image'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345402 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_api_ssl_compression = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345402 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_ca_certificates_file = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345402 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_catalog_info = image:glance:publicURL {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_num_retries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] glance_request_timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.345763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] goodness_function = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.346291 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] graceful_shutdown_timeout = 5 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.346291 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] group_api_class = cinder.group.api.API {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.346291 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] host = ncdv-master-1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.346291 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iet_conf = /etc/iet/ietd.conf {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.346291 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_compress_on_upload = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.349462 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_conversion_address_space_limit = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.349849 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_conversion_cpu_limit = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.350440 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.352831 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_upload_use_cinder_backend = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.353636 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_upload_use_internal_tenant = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.355150 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_volume_cache_enabled = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.355528 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_volume_cache_max_count = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.355899 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] image_volume_cache_max_size_gb = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.356284 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] instance_format = [instance: %(uuid)s] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.359411 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.359876 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iscsi_iotype = fileio {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.360279 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iscsi_secondary_ip_addresses = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.360659 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iscsi_target_flags = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.362065 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iscsi_write_cache = on {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.363114 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iser_helper = tgtadm {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.364505 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iser_ip_address = 10.222.0.20 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.366149 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iser_port = 3260 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.366644 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.367035 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_config_append = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.367408 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.367794 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_dir = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.368149 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_file = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.368448 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_options = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.368845 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_rotate_interval = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.369252 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.lockutils [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] Lock "cleanup_incomplete_backups_128544" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.002s {{(pid=129022) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Nov 13 20:55:11.369590 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] Cleaning up incomplete backup operations. Nov 13 20:55:11.372244 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_rotate_interval_type = days {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.373696 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] log_rotation_type = none {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.374023 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.376178 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.376668 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.377095 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.377451 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.377843 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] max_age = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.378237 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] max_logfile_count = 30 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.378623 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] max_logfile_size_mb = 200 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.379028 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] max_over_subscription_ratio = 20.0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.383432 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] message_reap_interval = 86400 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.387066 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] message_ttl = 2592000 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.387828 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 20:55:11.388316 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] monkey_patch = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.388731 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] monkey_patch_modules = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.389056 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] my_ip = 10.222.0.20 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.389426 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] no_snapshot_gb_quota = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.391709 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] num_iser_scan_tries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.392581 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] num_shell_tries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.393733 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] num_volume_device_scan_tries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.397816 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] nvmet_ns_id = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.397816 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] nvmet_port_id = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.398005 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_max_limit = 1000 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.398313 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_volume_ext_list = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.398763 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.399167 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_volume_listen = 0.0.0.0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.399554 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_volume_listen_port = 8776 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.399917 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_volume_use_ssl = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.400296 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] osapi_volume_workers = 2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.400654 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] per_volume_size_limit = -1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.404554 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] periodic_fuzzy_delay = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.407529 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] periodic_interval = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.407939 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] public_endpoint = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.413766 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] publish_errors = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.413766 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_backup_gigabytes = 1000 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.413766 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_backups = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.413766 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_consistencygroups = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.413766 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.413766 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_gigabytes = 1000 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.414530 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_groups = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.416764 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_snapshots = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.417168 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] quota_volumes = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.417653 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] rate_limit_burst = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.418044 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] rate_limit_except_level = CRITICAL {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.418418 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] rate_limit_interval = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.421851 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] replication_device = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.422243 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] report_discard_supported = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.422987 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] report_interval = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.423457 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] reservation_clean_interval = 86400 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.423773 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] reservation_expire = 86400 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.424314 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] reserved_percentage = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.425877 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.426268 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] restore_discard_excess_bytes = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.427928 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.428394 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] rpc_response_timeout = 120 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.428797 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] run_external_periodic_tasks = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.429186 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.429593 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] scst_target_driver = iscsi {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.433791 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] scst_target_iqn_name = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.434265 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_down_time = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.435453 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] snapshot_name_template = snapshot-%s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.435831 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] split_loggers = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.436145 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] state_path = /opt/stack/data/cinder {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.436537 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] storage_availability_zone = nova {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.436942 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] storage_protocol = iscsi {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.437316 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] syslog_log_facility = LOG_USER {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.437690 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] target_helper = lioadm {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.441138 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] target_ip_address = 10.222.0.20 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.441538 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] target_port = 3260 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.444978 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.445594 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] target_protocol = iscsi {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.445983 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] trace_flags = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.446363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] transfer_api_class = cinder.transfer.api.API {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.448223 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] transport_url = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.448597 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] until_refresh = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.448999 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_chap_auth = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.449368 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_default_quota_class = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.449732 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_eventlog = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.450094 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_journal = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.450453 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_json = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.450810 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_multipath_for_image_xfer = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.451172 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_stderr = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.451531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] use_syslog = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.458251 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] verify_glance_signatures = enabled {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.459079 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_api_class = cinder.volume.api.API {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.459509 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_backend_name = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.462419 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_clear = zero {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.462828 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_clear_ionice = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.463325 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_clear_size = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.463721 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.464298 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_copy_bps_limit = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.465446 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_dd_blocksize = 1M {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.471176 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.471176 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_name_template = volume-%s {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.471176 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volume_usage_audit_period = month {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.471176 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.471176 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] watch_log_file = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 13 20:55:11.471535 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.471822 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.enforce_new_defaults = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.472229 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.enforce_scope = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.472656 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.policy_default_rule = default {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.476820 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.477268 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.policy_file = policy.yaml {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.477698 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.478073 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.478565 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.478941 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.479304 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.479701 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_concurrency.disable_process_locking = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.484720 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.484720 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.admin_password = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.484720 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.484720 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.admin_token = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.484720 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.admin_user = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.484720 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_admin_prefix = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485247 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485247 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_port = 35357 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485247 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_protocol = https {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485247 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_section = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485247 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_type = password {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485247 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_uri = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.485591 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.auth_version = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.486435 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.cache = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.486713 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.487089 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.487453 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.delay_auth_decision = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.489375 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.489375 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.http_connect_timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.489375 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.489642 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.identity_uri = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.490500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.include_service_catalog = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.491472 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.insecure = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.491907 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.interface = public {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.492272 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.492649 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.493605 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.493800 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.494143 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.494795 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.495428 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.495848 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.496565 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.497038 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.497411 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.region_name = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.505774 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.505774 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.service_token_roles_required = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.505774 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.service_type = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.505774 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.token_cache_time = 300 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.505774 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.505774 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.backend = dogpile.cache.null {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.backend_argument = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.config_prefix = cache.oslo {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.debug_cache_backend = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.enabled = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.expiration_time = 600 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_dead_retry = 300 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506363 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_pool_maxsize = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.memcache_socket_timeout = 1.0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.proxies = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.506776 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.tls_allowed_ciphers = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.507178 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.tls_cafile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.507604 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.tls_certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.507814 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.tls_enabled = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.508173 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cache.tls_keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.508571 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] coordination.backend_url = etcd3+http://10.222.0.20:2379 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.509113 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.backend = sqlalchemy {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.509492 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.connection = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.509960 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.connection_debug = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.510256 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.connection_parameters = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.510636 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.connection_recycle_time = 3600 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.511065 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.connection_trace = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.511570 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.db_inc_retry_interval = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.512029 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.db_max_retries = 20 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.512556 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.db_max_retry_interval = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.512945 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.db_retry_interval = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.513694 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.max_overflow = 50 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.513924 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.max_pool_size = 5 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.514314 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.max_retries = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.514773 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.mysql_enable_ndb = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.515253 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.515765 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.pool_timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.516147 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.retry_interval = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.516920 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.slave_connection = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.517274 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.sqlite_synchronous = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.517702 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] database.use_db_reconnect = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.518095 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.518452 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.519096 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.519471 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.519833 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.520273 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.connection_string = messaging:// {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.520575 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.enabled = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.520962 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.es_doc_type = notification {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.521317 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.es_scroll_size = 10000 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.522523 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.es_scroll_time = 2m {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.522888 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.filter_error_trace = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.523255 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.hmac_keys = SECRET_KEY {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.523868 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.sentinel_service_name = mymaster {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.524294 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.socket_timeout = 0.1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.524650 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] profiler.trace_sqlalchemy = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.525213 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] privsep_osbrick.capabilities = [21] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.525860 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] privsep_osbrick.group = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.526221 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] privsep_osbrick.helper_command = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.526895 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.528122 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.528529 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] privsep_osbrick.user = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.529067 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.529500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_sys_admin.group = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.529938 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_sys_admin.helper_command = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.530347 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.530708 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.531312 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] cinder_sys_admin.user = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.531635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.auth_section = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.532060 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.auth_type = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.532459 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.cafile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.532837 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.533215 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.collect_timing = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.533834 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.insecure = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.534269 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.534701 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.send_service_user_token = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.535184 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.split_loggers = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.535647 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] service_user.timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.536210 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.backend_availability_zone = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.536646 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.chap_password = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.548864 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.chap_username = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.548864 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.548864 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.548864 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.548864 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.driver_client_cert = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.548864 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.driver_client_cert_key = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549390 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.driver_data_namespace = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549390 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549390 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549390 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.driver_use_ssl = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549390 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.enable_deferred_deletion = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549390 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.enable_unsupported_driver = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549757 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549757 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.filter_function = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549757 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.goodness_function = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549757 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549757 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.549757 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550165 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550165 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550165 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550165 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iscsi_iotype = fileio {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550165 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550165 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iscsi_target_flags = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iscsi_write_cache = on {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iser_helper = tgtadm {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iser_ip_address = 10.222.0.20 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iser_port = 3260 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550531 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550889 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.550889 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.num_shell_tries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.551444 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.551548 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.551992 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.nvmet_port_id = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.552346 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.556707 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rados_connection_interval = 5 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.556707 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rados_connection_retries = 3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.556707 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_ceph_conf = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.556707 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.556707 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.556707 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.557107 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.557107 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_pool = rbd {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.557107 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_secret_uuid = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.557452 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.557783 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.rbd_user = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.558141 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.558491 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.replication_device = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.558838 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.report_discard_supported = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.559183 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.559634 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.reserved_percentage = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.560003 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.scst_target_driver = iscsi {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.560352 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.scst_target_iqn_name = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.560732 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.storage_protocol = iscsi {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.561438 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.target_helper = tgtadm {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.561914 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.target_ip_address = 10.222.0.20 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.562278 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.target_port = 3260 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.562691 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.563116 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.target_protocol = iscsi {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.563465 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.trace_flags = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.563836 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.unique_fqdn_network = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.564505 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.use_chap_auth = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.564884 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.565324 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_backend_name = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.565910 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_clear = zero {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.566678 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_clear_ionice = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.567041 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_clear_size = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.571469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.571469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.571469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.571469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.571469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_reports.file_event_handler = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.571469 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.572500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_reports.log_dir = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.572500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.572500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.572500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.572500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.572500 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.573381 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.573381 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.573381 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.573781 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.576590 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.576590 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.576590 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.576590 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.576590 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.576590 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.577623 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.577623 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.577908 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.578108 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.578479 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.578949 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.579376 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.ssl = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.579891 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.580262 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.580653 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.581135 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_rabbit.ssl_version = {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.581544 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_notifications.driver = [] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.581979 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_notifications.retry = -1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.582451 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.583084 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] oslo_messaging_notifications.transport_url = **** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.583519 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.584008 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.585281 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.barbican_api_version = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.585748 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.barbican_endpoint = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.586283 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.barbican_endpoint_type = public {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.586655 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.barbican_region_name = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.587199 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.cafile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.587902 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.588318 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.collect_timing = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.588679 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.insecure = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.589052 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.589553 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.number_of_retries = 60 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.590514 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.retry_delay = 1 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.590880 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.send_service_user_token = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.591293 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.split_loggers = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.591646 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.591997 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.verify_ssl = True {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.592345 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican.verify_ssl_path = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.592715 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.auth_section = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.593279 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.auth_type = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.593631 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.cafile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.593975 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.594458 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.collect_timing = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.594816 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.insecure = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.595382 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.595833 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.split_loggers = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.596304 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] barbican_service_user.timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.596784 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.approle_role_id = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.597260 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.approle_secret_id = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.597633 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.cafile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.598199 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.certfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.collect_timing = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.insecure = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.keyfile = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.kv_mountpoint = secret {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.kv_version = 2 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.namespace = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.604562 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.root_token_id = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.605635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.split_loggers = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.605635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.ssl_ca_crt_file = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.605635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.timeout = None {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.605635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.use_ssl = False {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.605635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 13 20:55:11.605635 ncdv-master-1 cinder-backup[128544]: DEBUG oslo_service.service [None req-6ccc7258-a535-4fed-875c-9e11fb654df2 None None] ******************************************************************************** {{(pid=128544) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 13 20:55:11.621520 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.lockutils [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] Lock "cleanup_incomplete_backups_128544" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.256s {{(pid=129022) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Nov 13 20:55:11.633488 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 20:55:11.637559 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 20:55:11.680104 ncdv-master-1 cinder-backup[129022]: INFO cinder.keymgr.migration [None req-a5a53013-78d1-447f-b880-d5e12944cda3 None None] Not migrating encryption keys because the ConfKeyManager is still in use. Nov 13 20:55:11.697937 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.service [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] Creating RPC server for service cinder-backup {{(pid=129022) start /opt/stack/cinder/cinder/service.py:231}} Nov 13 20:55:11.706380 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.service [None req-a9f9e2ac-e481-4097-968f-9afdbc43c100 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=129022) start /opt/stack/cinder/cinder/service.py:237}} Nov 13 20:57:10.754440 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 20:57:10.755155 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 20:58:10.760116 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 20:58:10.760822 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 20:59:10.765304 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 20:59:10.765896 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:00:10.770297 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:00:10.770946 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:01:10.774760 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:01:10.775340 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:02:10.779703 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:02:10.780390 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:03:10.785105 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:03:10.785700 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:04:10.790722 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:04:10.791939 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:05:10.787389 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:05:10.787988 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:06:10.792944 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:06:10.798450 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:07:10.793098 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:07:10.793705 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:08:10.798646 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:08:10.799380 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:09:10.795303 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:09:10.797724 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:10:10.799820 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:10:10.799820 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:11:10.800131 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:11:10.801710 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:12:10.804731 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:12:10.806444 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:13:10.805657 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:13:10.806384 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:14:10.806249 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:14:10.806870 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:15:10.808329 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:15:10.809036 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:16:10.813704 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:16:10.816188 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:17:10.819275 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:17:10.819963 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:18:10.824368 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:18:10.825366 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:19:10.820802 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:19:10.821499 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:20:10.826284 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:20:10.827312 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:21:08.311101 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Create backup started, backup: 205b35ab-144d-48b5-b2bc-f0521892ad8a volume: 6c153bc1-b7bd-451c-8f93-eb637acb67b6 snapshot: 8b66b365-3b56-44a4-b89e-7ba19c51aaf3. Nov 13 21:21:08.475570 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:21:08.479371 ncdv-master-1 cinder-backup[129022]: INFO oslo.privsep.daemon [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmppqiyl3am/privsep.sock'] Nov 13 21:21:08.518878 ncdv-master-1 sudo[150300]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmppqiyl3am/privsep.sock Nov 13 21:21:08.524256 ncdv-master-1 sudo[150300]: pam_unix(sudo:session): session opened for user root by (uid=0) Nov 13 21:21:10.476251 ncdv-master-1 sudo[150300]: pam_unix(sudo:session): session closed for user root Nov 13 21:21:10.481482 ncdv-master-1 cinder-backup[129022]: INFO oslo.privsep.daemon [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Spawned new privsep daemon via rootwrap Nov 13 21:21:10.483027 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Accepted privsep connection to /tmp/tmppqiyl3am/privsep.sock {{(pid=129022) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}} Nov 13 21:21:10.484752 ncdv-master-1 cinder-backup[129022]: INFO oslo.privsep.daemon [-] privsep daemon starting Nov 13 21:21:10.485275 ncdv-master-1 cinder-backup[129022]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Nov 13 21:21:10.485756 ncdv-master-1 cinder-backup[129022]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none Nov 13 21:21:10.486234 ncdv-master-1 cinder-backup[129022]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 150304 Nov 13 21:21:10.491721 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b63badb-e9b2-4bde-939f-dee540de0f66]: (2,) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:10.797780 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:10.824832 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:21:10.825571 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:21:10.858229 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.060s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:21:10.858832 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[53dcb9f8-8330-4c4e-8773-04ecc0124f5d]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:10.862859 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:10.878724 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0b0877cc-093a-4fa7-a884-cd058ddaf3cb]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b0877cc-093a-4fa7-a884-cd058ddaf3cb]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:10.910129 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:10.940294 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6a4056bc-fb96-4cb5-9128-08ae98a91a7e]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:21:10.940758 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:21:10.941852 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a4056bc-fb96-4cb5-9128-08ae98a91a7e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:10.944631 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:10.989116 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.046s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:21:10.989660 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[e993a208-1057-427e-90c1-a1dcdd9c145b]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:10.991185 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:11.096159 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.104s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:21:11.096756 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[2771b857-5fc7-4507-b19a-de4609c517b3]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:11.101014 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:11.142109 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.042s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:21:11.142397 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[402ed3ea-71ed-418f-a33f-823b1b146b5f]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:21:11.143413 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:21:11.218443 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] CMD "nvme version" returned: 0 in 0.075s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:21:11.224727 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] <== get_connector_properties: return (2746ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:21:11.988098 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:21:17.644770 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:21:17.650542 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-ce7d4f75-34e1-45ad-ab42-18f443b63204', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ce7d4f75-34e1-45ad-ab42-18f443b63204', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:21:17.652998 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:21:17.952877 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] <== connect_volume: return (301ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:21:18.191018 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Starting backup of volume='6c153bc1-b7bd-451c-8f93-eb637acb67b6'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:21:18.290332 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:21:18.290793 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Creating backup base image='volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.base' for volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6. {{(pid=129022) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}} Nov 13 21:21:23.724328 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Copying data from volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6. {{(pid=129022) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}} Nov 13 21:21:24.085358 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferring data between 'volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6' and 'volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.base' {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}} Nov 13 21:21:24.086078 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] 8 chunks of 134217728 bytes to be transferred {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}} Nov 13 21:21:38.162988 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 1 of 8 (9311K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:21:51.276335 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 2 of 8 (9996K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:22:06.797686 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 3 of 8 (8444K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:22:10.822277 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:22:10.823858 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:22:20.560591 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 4 of 8 (9525K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:22:33.728240 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 5 of 8 (9952K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:22:48.133549 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 6 of 8 (9099K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:23:01.224112 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 7 of 8 (10013K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:23:10.825164 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:23:10.825825 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:23:14.582508 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Transferred chunk 8 of 8 (9812K/s) {{(pid=129022) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}} Nov 13 21:23:15.926161 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:23:15.942997 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:23:15.943412 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:23:15.972290 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:23:15.972290 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:23:15.972290 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] No metadata type 'volume-glance-metadata' available {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 13 21:23:15.972290 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Backing up metadata for volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:23:16.269275 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Backup '205b35ab-144d-48b5-b2bc-f0521892ad8a' of volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6 finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:23:16.269275 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-ce7d4f75-34e1-45ad-ab42-18f443b63204', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ce7d4f75-34e1-45ad-ab42-18f443b63204', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:23:16.691029 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] <== disconnect_volume: return (421ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:23:17.228909 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-e5c4ed04-d71a-4b77-b8a0-3ebe09368cfd tempest-VolumesSnapshotTestJSON-1404067749 None] Create backup finished. backup: 205b35ab-144d-48b5-b2bc-f0521892ad8a. Nov 13 21:23:17.789264 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Delete backup started, backup: 205b35ab-144d-48b5-b2bc-f0521892ad8a. Nov 13 21:23:17.792127 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Delete started for backup=205b35ab-144d-48b5-b2bc-f0521892ad8a {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:23:17.792909 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Trying diff format basename='volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.205b35ab-144d-48b5-b2bc-f0521892ad8a' for backup base image of volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:23:17.923234 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Image 'volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.205b35ab-144d-48b5-b2bc-f0521892ad8a' not found - trying diff format name {{(pid=129022) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}} Nov 13 21:23:18.242651 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Looking for snapshot of backup base 'volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.base' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:23:18.243253 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Backup base 'volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.base' has no snapshots {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}} Nov 13 21:23:18.243253 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] No backup snapshot to delete {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Nov 13 21:23:18.547302 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Deleting backup base image='volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.base' of volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6. Nov 13 21:23:22.011811 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:23:38.104939 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Base backup image='volume-6c153bc1-b7bd-451c-8f93-eb637acb67b6.backup.base' of volume 6c153bc1-b7bd-451c-8f93-eb637acb67b6 deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:23:38.353150 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Delete of backup '205b35ab-144d-48b5-b2bc-f0521892ad8a' for volume '6c153bc1-b7bd-451c-8f93-eb637acb67b6' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:23:38.381872 ncdv-master-1 cinder-backup[129022]: WARNING cinder.quota [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Deprecated: Default quota for resource: backups is set by the default quota flag: quota_backups, it is now deprecated. Please use the default quota class for default quota. Nov 13 21:23:38.382818 ncdv-master-1 cinder-backup[129022]: WARNING cinder.quota [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Deprecated: Default quota for resource: backup_gigabytes is set by the default quota flag: quota_backup_gigabytes, it is now deprecated. Please use the default quota class for default quota. Nov 13 21:23:38.491954 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Created reservations ['6e2b8164-e134-47c9-be8b-1cc555b84a26', '8315ea0f-7f3c-4d26-99d7-5107889e6765'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:23:38.743089 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-aedffe2d-e508-4f0c-9b9b-47bd627ae681 tempest-VolumesSnapshotTestJSON-1404067749 None] Delete backup finished, backup 205b35ab-144d-48b5-b2bc-f0521892ad8a deleted. Nov 13 21:24:10.821817 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:24:10.822546 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:25:10.826866 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:25:10.827633 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:26:10.832041 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:26:10.832904 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:27:10.828882 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:27:10.830289 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:28:10.826540 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:28:10.827167 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:29:10.832323 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:29:10.833157 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:30:10.829271 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:30:10.829934 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:30:44.801635 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Create backup started, backup: 68ec1da4-a799-4a52-8e9a-739ba233d718 volume: e378abcf-b451-4238-a036-755e32326427. Nov 13 21:30:44.923429 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:30:44.924931 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:44.944621 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:30:44.944977 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[59184a75-8b31-412b-a1e8-dde5b0a66d2e]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:30:44.946214 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:44.951538 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c21ce518-4b17-472a-9a4f-9a084da42e50]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:30:44.952155 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:30:44.953463 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[c21ce518-4b17-472a-9a4f-9a084da42e50]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:30:44.953757 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:44.958875 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0e9824d2-81f8-4df9-b0fe-e1b87325fe02]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:30:44.959429 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:30:44.960815 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e9824d2-81f8-4df9-b0fe-e1b87325fe02]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:30:44.961069 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:44.975984 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:30:44.976325 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[16512fac-b453-4743-bea7-f9222683c072]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:30:44.978146 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:44.986464 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:30:44.986789 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9cebcea-9789-488b-b646-6f1700539bc0]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:30:44.987927 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:44.995115 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:30:44.995448 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dd98e4f-7db4-4ac1-8908-3d03ebfdba70]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:30:44.996138 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:30:45.012864 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] CMD "nvme version" returned: 0 in 0.016s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:30:45.013969 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] <== get_connector_properties: return (89ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:30:45.821829 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:30:45.823233 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-e378abcf-b451-4238-a036-755e32326427', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e378abcf-b451-4238-a036-755e32326427', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:30:45.825984 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:30:46.088520 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] <== connect_volume: return (264ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:30:46.099407 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:30:46.210113 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Starting backup of volume='e378abcf-b451-4238-a036-755e32326427'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:30:46.211075 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Volume file is RBD: attempting optimized backup {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 13 21:30:46.268749 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Creating base image 'volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' {{(pid=129022) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 13 21:30:46.269276 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:30:47.565982 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Using --from-snap 'None' for incremental backup of volume e378abcf-b451-4238-a036-755e32326427. {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 13 21:30:47.566614 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Creating backup snapshot='backup.68ec1da4-a799-4a52-8e9a-739ba233d718.snap.1636839047.5659146' {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 13 21:30:50.363560 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Performing differential transfer from 'volume-e378abcf-b451-4238-a036-755e32326427' to 'volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:30:50.364234 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_ygspq61i --pool volumes volumes/volume-e378abcf-b451-4238-a036-755e32326427@backup.68ec1da4-a799-4a52-8e9a-739ba233d718.snap.1636839047.5659146 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:30:50.364395 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:30:54.805382 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Differential backup transfer completed in 4.4410s {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 13 21:30:54.806075 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:30:54.817924 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:30:54.822617 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:30:54.827589 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:30:54.827979 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:30:54.836364 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] No metadata type 'volume-glance-metadata' available {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 13 21:30:54.837528 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Backing up metadata for volume e378abcf-b451-4238-a036-755e32326427. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:30:55.123625 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Backup '68ec1da4-a799-4a52-8e9a-739ba233d718' of volume e378abcf-b451-4238-a036-755e32326427 finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:30:55.125557 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-e378abcf-b451-4238-a036-755e32326427', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'e378abcf-b451-4238-a036-755e32326427', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:30:56.255713 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] <== disconnect_volume: return (1129ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:30:56.478032 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-718c498d-032f-4c57-9cfd-d85938356b87 tempest-VolumesBackupsV39Test-1128315970 None] Create backup finished. backup: 68ec1da4-a799-4a52-8e9a-739ba233d718. Nov 13 21:30:57.505754 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Delete backup started, backup: 68ec1da4-a799-4a52-8e9a-739ba233d718. Nov 13 21:30:57.507482 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Delete started for backup=68ec1da4-a799-4a52-8e9a-739ba233d718 {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:30:57.507913 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Trying diff format basename='volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' for backup base image of volume e378abcf-b451-4238-a036-755e32326427. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:30:57.740867 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Looking for snapshot of backup base 'volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:30:57.741223 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Found snapshot 'backup.68ec1da4-a799-4a52-8e9a-739ba233d718.snap.1636839047.5659146' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:30:57.741670 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Deleting backup snapshot='backup.68ec1da4-a799-4a52-8e9a-739ba233d718.snap.1636839047.5659146' {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 13 21:31:00.076217 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:00.086224 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Create backup started, backup: 00abc107-9c6f-412d-9244-d0c53ba82ff6 volume: 7309cb3a-c938-400f-9ae4-c05d4eacaafc. Nov 13 21:31:00.151679 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:00.153051 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.159931 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:00.160258 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[561dbaf1-fc99-4422-a91a-845b66679073]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:00.161485 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.166918 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d5b7915f-5e6c-4ba0-8ae2-73d56da4cab0]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:31:00.167551 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:31:00.168975 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5b7915f-5e6c-4ba0-8ae2-73d56da4cab0]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:00.169398 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.175529 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f6fe2d4e-ab1d-41e1-8398-553906ab9940]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:31:00.176092 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:31:00.177306 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6fe2d4e-ab1d-41e1-8398-553906ab9940]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:00.177970 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.187247 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:00.187567 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[621e9f86-8a8f-4321-b5ca-898d3789ca5e]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:00.188605 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.196514 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:00.196855 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca440da0-a8df-4d45-8149-e83a82c557f6]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:00.197957 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.204766 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:00.205268 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[99f2d446-720a-43b3-bc24-fe90b019d471]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:00.206059 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:00.223759 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] CMD "nvme version" returned: 0 in 0.017s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:00.225560 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] <== get_connector_properties: return (72ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:00.938809 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:31:00.940237 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7309cb3a-c938-400f-9ae4-c05d4eacaafc', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:00.941288 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:31:01.210654 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] <== connect_volume: return (269ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:01.215629 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Deleting backup base image='volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' of volume e378abcf-b451-4238-a036-755e32326427. Nov 13 21:31:01.219355 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:01.296428 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Starting backup of volume='7309cb3a-c938-400f-9ae4-c05d4eacaafc'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:31:01.297610 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Volume file is RBD: attempting optimized backup {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 13 21:31:01.329841 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Creating base image 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' {{(pid=129022) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 13 21:31:01.330537 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:31:02.395245 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Using --from-snap 'None' for incremental backup of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc. {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 13 21:31:02.395865 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Creating backup snapshot='backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718' {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 13 21:31:04.880728 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Performing differential transfer from 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc' to 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:31:04.881340 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_n1rxlha3 --pool volumes volumes/volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc@backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:31:04.881669 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:31:08.100878 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Differential backup transfer completed in 3.2197s {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 13 21:31:08.101527 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:31:08.115206 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:31:08.115619 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:31:08.122577 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:31:08.122961 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:31:08.128868 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] No metadata type 'volume-glance-metadata' available {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 13 21:31:08.129400 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Backing up metadata for volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:31:08.315126 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Backup '00abc107-9c6f-412d-9244-d0c53ba82ff6' of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:31:08.316736 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7309cb3a-c938-400f-9ae4-c05d4eacaafc', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:09.218174 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] <== disconnect_volume: return (900ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:09.220570 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Base backup image='volume-e378abcf-b451-4238-a036-755e32326427.backup.68ec1da4-a799-4a52-8e9a-739ba233d718' of volume e378abcf-b451-4238-a036-755e32326427 deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:31:09.414346 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-57c2da14-a084-49b7-8e00-094e73425f44 tempest-VolumesBackupsAdminTest-1301942349 None] Create backup finished. backup: 00abc107-9c6f-412d-9244-d0c53ba82ff6. Nov 13 21:31:09.671911 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Delete of backup '68ec1da4-a799-4a52-8e9a-739ba233d718' for volume 'e378abcf-b451-4238-a036-755e32326427' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:31:09.791131 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Created reservations ['96d23247-b4cf-41fa-9281-cc8f36f86586', '57befc96-98bc-4601-997c-ae89f0a468a2'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:31:10.004234 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-aad627d4-68d5-4d97-a710-e160f17a9137 tempest-VolumesBackupsV39Test-1128315970 None] Delete backup finished, backup 68ec1da4-a799-4a52-8e9a-739ba233d718 deleted. Nov 13 21:31:10.494610 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-76a11973-5aa4-41bd-a960-b7560c5edf68 tempest-VolumesBackupsAdminTest-2044406936 None] Export record started, backup: 00abc107-9c6f-412d-9244-d0c53ba82ff6. Nov 13 21:31:10.497872 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-76a11973-5aa4-41bd-a960-b7560c5edf68 tempest-VolumesBackupsAdminTest-2044406936 None] Export record finished, backup 00abc107-9c6f-412d-9244-d0c53ba82ff6 exported. Nov 13 21:31:10.826144 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:31:10.826530 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:31:11.113801 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-429d2dab-614e-420b-a660-f8151b0e86cc tempest-VolumesBackupsAdminTest-2044406936 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiZjg5ZjU0MWQtZDU0Mi00YzE2LTljZmYtNDYyMmE3MDc3Njg0IiwgInVzZXJfaWQiOiAiNzZmYjRlYTE1NDhkNDAxZmFmZDlmN2M5Y2FhZWY1YjUiLCAicHJvamVjdF9pZCI6ICJlNTg3YzQ3Y2U0Nzk0N2I1OTdiNzRmN2VlMTkwMDYxYyIsICJ2b2x1bWVfaWQiOiAiNzMwOWNiM2EtYzkzOC00MDBmLTlhZTQtYzA1ZDRlYWNhYWZjIiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC03NDgwNDgwNjAiLCAiZGlzcGxheV9kZXNjcmlwdGlvbiI6IG51bGwsICJzZXJ2aWNlX21ldGFkYXRhIjogIntcImJhc2VcIjogXCJ2b2x1bWUtNzMwOWNiM2EtYzkzOC00MDBmLTlhZTQtYzA1ZDRlYWNhYWZjLmJhY2t1cC4wMGFiYzEwNy05YzZmLTQxMmQtOTI0NC1kMGM1M2JhODJmZjZcIn0iLCAic2VydmljZSI6ICJjaW5kZXIuYmFja3VwLmRyaXZlcnMuY2VwaC5DZXBoQmFja3VwRHJpdmVyIiwgIm9iamVjdF9jb3VudCI6IDAsICJ0ZW1wX3ZvbHVtZV9pZCI6IG51bGwsICJ0ZW1wX3NuYXBzaG90X2lkIjogbnVsbCwgIm51bV9kZXBlbmRlbnRfYmFja3VwcyI6IDAsICJzbmFwc2hvdF9pZCI6IG51bGwsICJkYXRhX3RpbWVzdGFtcCI6ICIyMDIxLTExLTEzVDIxOjMxOjAwWiIsICJyZXN0b3JlX3ZvbHVtZV9pZCI6IG51bGwsICJtZXRhZGF0YSI6IHt9LCAiZW5jcnlwdGlvbl9rZXlfaWQiOiBudWxsLCAiY3JlYXRlZF9hdCI6ICIyMDIxLTExLTEzVDIxOjMxOjAwWiIsICJ1cGRhdGVkX2F0IjogIjIwMjEtMTEtMTNUMjE6MzE6MDlaIiwgImRlbGV0ZWRfYXQiOiBudWxsLCAiZGVsZXRlZCI6IGZhbHNlfQ==. Nov 13 21:31:11.122937 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-429d2dab-614e-420b-a660-f8151b0e86cc tempest-VolumesBackupsAdminTest-2044406936 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:11.427626 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-429d2dab-614e-420b-a660-f8151b0e86cc tempest-VolumesBackupsAdminTest-2044406936 None] Import record id f89f541d-d542-4c16-9cff-4622a7077684 metadata from driver finished. Nov 13 21:31:17.047719 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Restore backup started, backup: 00abc107-9c6f-412d-9244-d0c53ba82ff6 volume: 6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9. Nov 13 21:31:17.099400 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:17.106080 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.122922 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:17.123359 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbd8aba0-a89c-4557-8cd9-9980e31c7726]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:17.126183 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.132742 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[69d7123e-9bb8-4087-aaea-14442d876496]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:31:17.133332 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:31:17.134882 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[69d7123e-9bb8-4087-aaea-14442d876496]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:17.137041 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.145255 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[242d5cc7-6181-4ac2-ae5f-4604b60dcca2]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:31:17.145590 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:31:17.147704 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[242d5cc7-6181-4ac2-ae5f-4604b60dcca2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:17.152774 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.182726 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.029s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:17.183291 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[175b6032-67a2-4bdf-9063-9f6150a751da]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:17.185341 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.201438 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.016s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:17.204548 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5adc5c0-24bd-4a4b-8320-d743a6aea043]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:17.204548 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.219146 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.015s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:17.219431 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3c580db-7440-48fc-aa1c-864a6186b0e5]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:17.220355 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:17.250888 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] CMD "nvme version" returned: 0 in 0.030s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:17.252392 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] <== get_connector_properties: return (152ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:17.966218 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:31:17.981693 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:17.982326 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:31:18.193792 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] <== connect_volume: return (225ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:18.204649 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:18.212596 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Starting restore from Ceph backup=00abc107-9c6f-412d-9244-d0c53ba82ff6 to volume=volume-6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9 {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 13 21:31:18.273355 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Looking for snapshot of backup base 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:31:18.273752 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Found snapshot 'backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:31:18.278379 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Volume file is RBD. {{(pid=129022) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 13 21:31:18.310644 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Attempting differential restore. {{(pid=129022) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 13 21:31:18.311124 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Attempting incremental restore from base='volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' snap='backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718' {{(pid=129022) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 13 21:31:18.311511 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Performing differential transfer from 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' to 'volume-6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:31:18.311902 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6@backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:31:18.312271 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_ajlwj_7f --pool volumes - volumes/volume-6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:31:21.222570 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Restore transfer completed in 2.9105s {{(pid=129022) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 13 21:31:21.223615 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 13 21:31:21.279581 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Restoring volume base metadata {{(pid=129022) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 13 21:31:21.393530 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] No metadata of type 'volume-metadata' to restore {{(pid=129022) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 13 21:31:21.393906 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=129022) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 13 21:31:21.397184 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Restore to volume 6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9 finished successfully. {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 13 21:31:21.398725 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:21.562033 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] <== disconnect_volume: return (162ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:21.608814 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.manager [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Encryption key ID for volume 6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9 already matches encryption key ID in backup 00abc107-9c6f-412d-9244-d0c53ba82ff6. {{(pid=129022) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 13 21:31:21.828329 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-5599a462-7f34-4b40-9454-61a748031090 tempest-VolumesBackupsAdminTest-1301942349 None] Finished restoring backup 00abc107-9c6f-412d-9244-d0c53ba82ff6 to volume 6e5a78f2-6fb5-4d7a-a91e-5ec97a8e40e9. Nov 13 21:31:22.785615 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Delete backup started, backup: f89f541d-d542-4c16-9cff-4622a7077684. Nov 13 21:31:22.788434 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Delete started for backup=f89f541d-d542-4c16-9cff-4622a7077684 {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:31:22.788798 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Trying diff format basename='volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' for backup base image of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:31:23.096112 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Looking for snapshot of backup base 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:31:23.096627 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Backup 'f89f541d-d542-4c16-9cff-4622a7077684' has no snapshot {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}} Nov 13 21:31:23.097201 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] No backup snapshot to delete {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}} Nov 13 21:31:23.326732 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Backup base image of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc still has 1 snapshots so skipping base image delete. Nov 13 21:31:23.580003 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Metadata backup object 'backup.f89f541d-d542-4c16-9cff-4622a7077684.meta' not found - ignoring {{(pid=129022) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}} Nov 13 21:31:23.586344 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Delete of backup 'f89f541d-d542-4c16-9cff-4622a7077684' for volume '7309cb3a-c938-400f-9ae4-c05d4eacaafc' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:31:23.591965 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:23.746885 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Created reservations ['ebe816a5-c86a-4ba5-8c82-b813b88d2b8f', '8253d122-2bf9-4eec-918b-c01d6532301c'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:31:23.997036 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-ea073c09-99b6-4307-b272-b433fd6f03f1 tempest-VolumesBackupsAdminTest-2044406936 None] Delete backup finished, backup f89f541d-d542-4c16-9cff-4622a7077684 deleted. Nov 13 21:31:24.988237 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Delete backup started, backup: 00abc107-9c6f-412d-9244-d0c53ba82ff6. Nov 13 21:31:24.992373 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Delete started for backup=00abc107-9c6f-412d-9244-d0c53ba82ff6 {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:31:24.992373 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Trying diff format basename='volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' for backup base image of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:31:25.204038 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Looking for snapshot of backup base 'volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:31:25.205138 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Found snapshot 'backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:31:25.205516 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Deleting backup snapshot='backup.00abc107-9c6f-412d-9244-d0c53ba82ff6.snap.1636839062.3952718' {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 13 21:31:28.590488 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Deleting backup base image='volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc. Nov 13 21:31:32.093558 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:36.842286 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Base backup image='volume-7309cb3a-c938-400f-9ae4-c05d4eacaafc.backup.00abc107-9c6f-412d-9244-d0c53ba82ff6' of volume 7309cb3a-c938-400f-9ae4-c05d4eacaafc deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:31:37.087058 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Delete of backup '00abc107-9c6f-412d-9244-d0c53ba82ff6' for volume '7309cb3a-c938-400f-9ae4-c05d4eacaafc' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:31:37.169623 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Created reservations ['3df0cd6d-6ba8-48ec-bcd4-24256619f2de', 'b4f57854-78b9-4fd5-99f0-98798b775605'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:31:37.445382 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-620a3814-916e-401c-821b-c680a3b2b9bf tempest-VolumesBackupsAdminTest-1301942349 None] Delete backup finished, backup 00abc107-9c6f-412d-9244-d0c53ba82ff6 deleted. Nov 13 21:31:40.915739 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Create backup started, backup: 6df6f387-9280-42f1-9f84-d5a5e3e96312 volume: f3e95b9a-c4bb-4361-967e-b96869e60543. Nov 13 21:31:40.988200 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:40.989642 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:40.997595 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:40.997977 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdb97bb1-26c9-4f53-86bc-44b49d41f3d8]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:40.999253 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:41.006880 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c974fffd-b546-4fbd-95cb-ee58f54f044b]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:31:41.007320 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:31:41.008754 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[c974fffd-b546-4fbd-95cb-ee58f54f044b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:41.009823 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:41.016776 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[759339b5-f625-4945-8397-d921edef2bac]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:31:41.017448 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:31:41.019123 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[759339b5-f625-4945-8397-d921edef2bac]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:41.020854 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:41.030925 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.010s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:41.031414 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f81e408-9fe0-42d9-9c42-2ce77296156c]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:41.034801 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:41.046626 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.012s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:41.054166 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[acf7d6f3-bea6-4212-9397-a9348ab5843c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:41.054166 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:41.056626 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:41.060569 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[388bdfe1-37e4-4c0b-a216-3222b37b1eb1]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:31:41.061433 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:31:41.081329 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:31:41.088010 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] <== get_connector_properties: return (98ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:41.917416 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:31:41.918958 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-f3e95b9a-c4bb-4361-967e-b96869e60543', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f3e95b9a-c4bb-4361-967e-b96869e60543', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:41.919972 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:31:42.124595 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] <== connect_volume: return (205ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:42.138658 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:31:42.222916 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Starting backup of volume='f3e95b9a-c4bb-4361-967e-b96869e60543'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:31:42.223852 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Volume file is RBD: attempting optimized backup {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 13 21:31:42.247637 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Creating base image 'volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' {{(pid=129022) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 13 21:31:42.248142 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:31:43.461214 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Using --from-snap 'None' for incremental backup of volume f3e95b9a-c4bb-4361-967e-b96869e60543. {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 13 21:31:43.461214 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Creating backup snapshot='backup.6df6f387-9280-42f1-9f84-d5a5e3e96312.snap.1636839103.4608097' {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 13 21:31:46.288738 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Performing differential transfer from 'volume-f3e95b9a-c4bb-4361-967e-b96869e60543' to 'volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:31:46.289380 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_xs0u6jhq --pool volumes volumes/volume-f3e95b9a-c4bb-4361-967e-b96869e60543@backup.6df6f387-9280-42f1-9f84-d5a5e3e96312.snap.1636839103.4608097 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:31:46.289533 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:31:49.559567 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Differential backup transfer completed in 3.2704s {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 13 21:31:49.560196 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:31:49.573603 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:31:49.574025 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:31:49.580465 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:31:49.580893 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:31:49.586969 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] No metadata type 'volume-glance-metadata' available {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 13 21:31:49.587516 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Backing up metadata for volume f3e95b9a-c4bb-4361-967e-b96869e60543. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:31:50.020462 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Backup '6df6f387-9280-42f1-9f84-d5a5e3e96312' of volume f3e95b9a-c4bb-4361-967e-b96869e60543 finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:31:50.022083 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-f3e95b9a-c4bb-4361-967e-b96869e60543', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f3e95b9a-c4bb-4361-967e-b96869e60543', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:31:50.944591 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] <== disconnect_volume: return (920ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:31:51.249690 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-fa1f9d9b-63ac-442b-ae88-232e3edbe848 tempest-VolumesBackupsAdminTest-1301942349 None] Create backup finished. backup: 6df6f387-9280-42f1-9f84-d5a5e3e96312. Nov 13 21:31:52.416755 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-2902db3f-3ebc-4b03-a315-39a87a6779a1 tempest-VolumesBackupsAdminTest-2044406936 None] Reset backup status started, backup_id: 6df6f387-9280-42f1-9f84-d5a5e3e96312, status: error. Nov 13 21:31:52.417338 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-2902db3f-3ebc-4b03-a315-39a87a6779a1 tempest-VolumesBackupsAdminTest-2044406936 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver. Nov 13 21:31:53.583632 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Delete backup started, backup: 6df6f387-9280-42f1-9f84-d5a5e3e96312. Nov 13 21:31:53.587973 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Delete started for backup=6df6f387-9280-42f1-9f84-d5a5e3e96312 {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:31:53.588479 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Trying diff format basename='volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' for backup base image of volume f3e95b9a-c4bb-4361-967e-b96869e60543. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:31:54.018310 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Looking for snapshot of backup base 'volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:31:54.019476 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Found snapshot 'backup.6df6f387-9280-42f1-9f84-d5a5e3e96312.snap.1636839103.4608097' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:31:54.019476 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Deleting backup snapshot='backup.6df6f387-9280-42f1-9f84-d5a5e3e96312.snap.1636839103.4608097' {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 13 21:31:57.747900 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Deleting backup base image='volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' of volume f3e95b9a-c4bb-4361-967e-b96869e60543. Nov 13 21:32:02.143272 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:32:05.943647 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Base backup image='volume-f3e95b9a-c4bb-4361-967e-b96869e60543.backup.6df6f387-9280-42f1-9f84-d5a5e3e96312' of volume f3e95b9a-c4bb-4361-967e-b96869e60543 deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:32:06.109734 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Delete of backup '6df6f387-9280-42f1-9f84-d5a5e3e96312' for volume 'f3e95b9a-c4bb-4361-967e-b96869e60543' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:32:06.218119 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Created reservations ['e70e3e0c-9845-41d7-93e4-663b17fa1d9d', 'b435b0f9-7e18-45f2-a14d-0659c22f4b8b'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:32:06.457352 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-4ab947d1-4bca-4aac-ad1a-2db8e7eb5f36 tempest-VolumesBackupsAdminTest-1301942349 None] Delete backup finished, backup 6df6f387-9280-42f1-9f84-d5a5e3e96312 deleted. Nov 13 21:32:10.832195 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:32:10.832841 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:33:10.844902 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:33:10.844902 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:34:10.834176 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:34:10.834932 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:34:13.248294 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Create backup started, backup: 929d00cf-2b10-4765-899c-662d1c38ac0f volume: 410acec1-b74d-4a4c-a7cd-44223533cde2. Nov 13 21:34:13.347126 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:34:13.349127 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.363600 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:34:13.363941 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f3307fd-e6f8-4323-b3aa-28e2beff034f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:34:13.366402 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.379613 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[bed8b493-fea6-4bc2-ba4b-c9b37bfe083d]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:34:13.381232 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:34:13.385003 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[bed8b493-fea6-4bc2-ba4b-c9b37bfe083d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:34:13.388037 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.402050 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[169f35d3-2c15-4453-b865-52d4772ff5e6]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:34:13.404530 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[169f35d3-2c15-4453-b865-52d4772ff5e6]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:34:13.407000 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.415866 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:34:13.416264 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d4de719-10d3-4091-b695-5e1d542def93]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:34:13.417452 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.426294 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:34:13.426722 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[4aa679c0-5b82-4cf7-a801-3b9a229690c6]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:34:13.428144 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.434999 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:34:13.436160 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[66b9a76b-49cb-4be4-a249-ec7a4a86a275]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:34:13.437546 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:34:13.456583 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:34:13.458227 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] <== get_connector_properties: return (110ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:34:14.193210 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:34:14.194622 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-410acec1-b74d-4a4c-a7cd-44223533cde2', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '410acec1-b74d-4a4c-a7cd-44223533cde2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:34:14.195677 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:34:14.550334 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] <== connect_volume: return (354ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:34:14.567505 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:34:14.677533 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Starting backup of volume='410acec1-b74d-4a4c-a7cd-44223533cde2'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:34:14.678660 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Volume file is RBD: attempting optimized backup {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 13 21:34:14.719484 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Creating base image 'volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' {{(pid=129022) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 13 21:34:14.719986 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:34:15.570973 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Using --from-snap 'None' for incremental backup of volume 410acec1-b74d-4a4c-a7cd-44223533cde2. {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 13 21:34:15.571653 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Creating backup snapshot='backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175' {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 13 21:34:17.832070 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Performing differential transfer from 'volume-410acec1-b74d-4a4c-a7cd-44223533cde2' to 'volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:34:17.832677 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_v3utvu15 --pool volumes volumes/volume-410acec1-b74d-4a4c-a7cd-44223533cde2@backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:34:17.832859 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:35:10.839703 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:35:10.840404 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:35:11.593748 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Differential backup transfer completed in 53.7616s {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 13 21:35:11.594343 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:35:11.606656 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:35:11.611518 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:35:11.613486 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:35:11.614527 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:35:11.621741 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 13 21:35:11.622334 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Backing up metadata for volume 410acec1-b74d-4a4c-a7cd-44223533cde2. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:35:11.856922 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Backup '929d00cf-2b10-4765-899c-662d1c38ac0f' of volume 410acec1-b74d-4a4c-a7cd-44223533cde2 finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:35:11.859153 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-410acec1-b74d-4a4c-a7cd-44223533cde2', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '410acec1-b74d-4a4c-a7cd-44223533cde2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:35:12.768301 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] <== disconnect_volume: return (907ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:35:14.808029 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-1df3a163-b80c-4551-a3c2-d7059d3c51b2 tempest-TestVolumeBackupRestore-355465777 None] Create backup finished. backup: 929d00cf-2b10-4765-899c-662d1c38ac0f. Nov 13 21:35:18.796455 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Restore backup started, backup: 929d00cf-2b10-4765-899c-662d1c38ac0f volume: 86820b45-ab53-44db-896f-e5f387a2ced9. Nov 13 21:35:18.809947 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:35:18.811360 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.822501 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:35:18.823709 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc51632a-829e-4cd4-8774-e49c297b54d9]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:35:18.824529 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.830176 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[650bb1f5-ec50-4912-bf17-c5f6a7d3dda6]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:35:18.830797 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:35:18.832150 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[650bb1f5-ec50-4912-bf17-c5f6a7d3dda6]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:35:18.832379 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.838870 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6701e7b2-a3bf-4cc2-abb9-d97487c8f2fb]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:35:18.839409 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:35:18.840800 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[6701e7b2-a3bf-4cc2-abb9-d97487c8f2fb]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:35:18.842663 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.855691 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:35:18.856027 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[feeda9fb-615a-4de3-8cf5-bf74f9b5c427]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:35:18.857239 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.867112 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:35:18.867429 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[237a8d88-8753-44f5-a3f5-93b157b42393]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:35:18.868483 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.875160 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:35:18.875498 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[f16f7967-7c69-4d6a-9456-5373b308932b]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:35:18.881089 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:35:18.894230 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:35:18.895275 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] <== get_connector_properties: return (85ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:35:19.702027 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:35:19.703492 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-86820b45-ab53-44db-896f-e5f387a2ced9', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '86820b45-ab53-44db-896f-e5f387a2ced9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:35:19.704496 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:35:20.029380 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] <== connect_volume: return (325ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:35:20.036878 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:35:20.048847 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Starting restore from Ceph backup=929d00cf-2b10-4765-899c-662d1c38ac0f to volume=volume-86820b45-ab53-44db-896f-e5f387a2ced9 {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 13 21:35:20.158930 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Looking for snapshot of backup base 'volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:35:20.159323 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Found snapshot 'backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:35:20.164161 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Volume file is RBD. {{(pid=129022) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 13 21:35:20.195238 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Attempting differential restore. {{(pid=129022) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 13 21:35:20.195718 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Attempting incremental restore from base='volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' snap='backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175' {{(pid=129022) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 13 21:35:20.196272 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Performing differential transfer from 'volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' to 'volume-86820b45-ab53-44db-896f-e5f387a2ced9' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:35:20.198207 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f@backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:35:20.198686 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_90c17hbl --pool volumes - volumes/volume-86820b45-ab53-44db-896f-e5f387a2ced9' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:36:06.429744 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Restore transfer completed in 46.2329s {{(pid=129022) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 13 21:36:06.430736 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 13 21:36:06.461285 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Restoring volume base metadata {{(pid=129022) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 13 21:36:06.568817 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] No metadata of type 'volume-metadata' to restore {{(pid=129022) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 13 21:36:06.569204 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Restoring volume glance metadata {{(pid=129022) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 13 21:36:07.627208 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Restore to volume 86820b45-ab53-44db-896f-e5f387a2ced9 finished successfully. {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 13 21:36:07.628776 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-86820b45-ab53-44db-896f-e5f387a2ced9', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '86820b45-ab53-44db-896f-e5f387a2ced9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:36:07.749646 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] <== disconnect_volume: return (120ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:36:07.799079 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.manager [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Encryption key ID for volume 86820b45-ab53-44db-896f-e5f387a2ced9 already matches encryption key ID in backup 929d00cf-2b10-4765-899c-662d1c38ac0f. {{(pid=129022) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 13 21:36:08.025592 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-70726fef-ae14-4acc-beeb-9af9e334935a tempest-TestVolumeBackupRestore-355465777 None] Finished restoring backup 929d00cf-2b10-4765-899c-662d1c38ac0f to volume 86820b45-ab53-44db-896f-e5f387a2ced9. Nov 13 21:36:10.845308 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:36:10.855129 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:37:10.850924 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:37:10.853700 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:38:10.849471 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:38:10.850511 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:38:40.420271 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Delete backup started, backup: 929d00cf-2b10-4765-899c-662d1c38ac0f. Nov 13 21:38:40.422558 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Delete started for backup=929d00cf-2b10-4765-899c-662d1c38ac0f {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:38:40.422987 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Trying diff format basename='volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' for backup base image of volume 410acec1-b74d-4a4c-a7cd-44223533cde2. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:38:40.616098 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Looking for snapshot of backup base 'volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:38:40.617034 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Found snapshot 'backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:38:40.617932 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Deleting backup snapshot='backup.929d00cf-2b10-4765-899c-662d1c38ac0f.snap.1636839255.5710175' {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 13 21:38:42.824403 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:38:44.342786 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Deleting backup base image='volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' of volume 410acec1-b74d-4a4c-a7cd-44223533cde2. Nov 13 21:38:55.958317 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Base backup image='volume-410acec1-b74d-4a4c-a7cd-44223533cde2.backup.929d00cf-2b10-4765-899c-662d1c38ac0f' of volume 410acec1-b74d-4a4c-a7cd-44223533cde2 deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:38:56.201192 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Delete of backup '929d00cf-2b10-4765-899c-662d1c38ac0f' for volume '410acec1-b74d-4a4c-a7cd-44223533cde2' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:38:56.374138 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Created reservations ['102d2944-352b-460c-8d49-d5dd35e9d3b4', 'ecbef8f7-2213-4706-ba2c-ea9bf0c19825'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:38:56.688287 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-badc3d66-8256-47d0-b523-bcfb1bf28415 tempest-TestVolumeBackupRestore-355465777 None] Delete backup finished, backup 929d00cf-2b10-4765-899c-662d1c38ac0f deleted. Nov 13 21:39:10.853651 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:39:10.854891 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:40:10.846546 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:40:10.847595 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:41:10.843862 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:41:10.845711 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:41:23.931099 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Create backup started, backup: a1874310-9ccd-4a97-9de8-3493c3214bbc volume: ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522. Nov 13 21:41:24.000873 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:41:24.003358 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.027896 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.024s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:41:24.032009 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3fbf85d-1f03-4c90-946e-de8c7a9ee4a2]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:41:24.035077 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.048741 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6d5f7e8e-a58d-4b68-a546-309cb6a84f85]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:41:24.049954 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:41:24.051220 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d5f7e8e-a58d-4b68-a546-309cb6a84f85]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:41:24.054770 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.074492 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[26316f63-eab9-4c38-8357-84fc5a4cc78b]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:41:24.075174 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:41:24.076618 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[26316f63-eab9-4c38-8357-84fc5a4cc78b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:41:24.078322 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.107310 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.029s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:41:24.107639 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[af687e1a-eecf-448f-98db-eaf203fa6af7]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:41:24.108789 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.139320 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.030s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:41:24.139732 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ec20d25-b739-4221-bb93-bd3e479dd2f8]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:41:24.142057 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.164985 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.023s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:41:24.165198 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4dcf50c-00a2-4a11-956c-d4414d3e6ca9]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:41:24.166130 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:41:24.221045 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] CMD "nvme version" returned: 0 in 0.052s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:41:24.221045 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] <== get_connector_properties: return (218ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:41:25.389411 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:41:25.391000 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:41:25.392329 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:41:25.729079 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] <== connect_volume: return (336ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:41:25.739089 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:41:25.819220 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Starting backup of volume='ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:41:25.823714 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Volume file is RBD: attempting optimized backup {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 13 21:41:25.917792 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Creating base image 'volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' {{(pid=129022) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 13 21:41:25.918290 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:41:26.807301 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Using --from-snap 'None' for incremental backup of volume ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522. {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 13 21:41:26.808557 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Creating backup snapshot='backup.a1874310-9ccd-4a97-9de8-3493c3214bbc.snap.1636839686.8081045' {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 13 21:41:29.097393 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Performing differential transfer from 'volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522' to 'volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:41:29.097976 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_z4jaodan --pool volumes volumes/volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522@backup.a1874310-9ccd-4a97-9de8-3493c3214bbc.snap.1636839686.8081045 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:41:29.098275 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:41:33.213588 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Differential backup transfer completed in 4.1169s {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 13 21:41:33.214238 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:41:33.227604 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:41:33.231147 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:41:33.239952 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:41:33.241361 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:41:33.248255 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] No metadata type 'volume-glance-metadata' available {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 13 21:41:33.249165 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Backing up metadata for volume ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:41:33.438799 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Backup 'a1874310-9ccd-4a97-9de8-3493c3214bbc' of volume ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522 finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:41:33.440612 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:41:33.757592 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] <== disconnect_volume: return (316ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:41:33.984803 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-288d7269-fbcc-44c0-9df9-71fcc37dd610 tempest-VolumesBackupsTest-1116982884 None] Create backup finished. backup: a1874310-9ccd-4a97-9de8-3493c3214bbc. Nov 13 21:41:34.415199 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Delete backup started, backup: a1874310-9ccd-4a97-9de8-3493c3214bbc. Nov 13 21:41:34.417227 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Delete started for backup=a1874310-9ccd-4a97-9de8-3493c3214bbc {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:41:34.417655 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Trying diff format basename='volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' for backup base image of volume ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:41:34.655859 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Looking for snapshot of backup base 'volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:41:34.656393 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Found snapshot 'backup.a1874310-9ccd-4a97-9de8-3493c3214bbc.snap.1636839686.8081045' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:41:34.657982 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Deleting backup snapshot='backup.a1874310-9ccd-4a97-9de8-3493c3214bbc.snap.1636839686.8081045' {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 13 21:41:37.911620 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Deleting backup base image='volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' of volume ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522. Nov 13 21:41:42.838659 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:41:44.398001 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Base backup image='volume-ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522.backup.a1874310-9ccd-4a97-9de8-3493c3214bbc' of volume ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522 deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:41:44.680455 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Delete of backup 'a1874310-9ccd-4a97-9de8-3493c3214bbc' for volume 'ec8c7562-eaeb-4b75-9d3e-1fbc19ef5522' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:41:44.768937 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Created reservations ['5f361ef4-76f3-4778-8b4b-4c1283c3fa88', 'c3a6ada7-9841-4e93-9a10-a8643b371c2a'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:41:44.979243 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-8c471015-f59b-4975-a94e-8537f794972d tempest-VolumesBackupsTest-1116982884 None] Delete backup finished, backup a1874310-9ccd-4a97-9de8-3493c3214bbc deleted. Nov 13 21:42:10.629282 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Create backup started, backup: a621b34b-2c3b-42af-a81a-af3652e012a3 volume: 6aa0150e-3650-4ac5-8171-75ce1947688f. Nov 13 21:42:10.769470 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:42:10.771749 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.779085 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:42:10.779484 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[4121ecb6-bf27-424e-ab79-f28316279deb]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:42:10.780837 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.786286 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f29acfc2-4173-4e8d-b702-90718e55f190]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:42:10.786834 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:42:10.788354 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[f29acfc2-4173-4e8d-b702-90718e55f190]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:42:10.793041 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.796849 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[41930d62-92cd-449d-966d-c5600a77f262]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:42:10.798055 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:42:10.799694 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[41930d62-92cd-449d-966d-c5600a77f262]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:42:10.801785 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.810658 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:42:10.811260 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7953741-2ddc-4e9c-b358-42592b64ae1a]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:42:10.812638 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.821336 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:42:10.821661 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[18307952-68e7-484a-9e0e-02613b645eee]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:42:10.823752 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.829609 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:42:10.830112 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e791192-7539-4ab4-9c6e-7f765c05f500]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:42:10.830820 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:42:10.849104 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:42:10.849511 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:42:10.860629 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] CMD "nvme version" returned: 0 in 0.026s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:42:10.860629 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] <== get_connector_properties: return (87ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:42:11.590631 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:42:11.592121 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-6aa0150e-3650-4ac5-8171-75ce1947688f', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6aa0150e-3650-4ac5-8171-75ce1947688f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:42:11.601454 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:42:11.991112 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] <== connect_volume: return (398ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:42:11.999477 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:42:12.075599 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Starting backup of volume='6aa0150e-3650-4ac5-8171-75ce1947688f'. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}} Nov 13 21:42:12.075599 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Volume file is RBD: attempting optimized backup {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}} Nov 13 21:42:12.119043 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Creating base image 'volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' {{(pid=129022) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}} Nov 13 21:42:12.119545 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=129022) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}} Nov 13 21:42:13.153751 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Using --from-snap 'None' for incremental backup of volume 6aa0150e-3650-4ac5-8171-75ce1947688f. {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}} Nov 13 21:42:13.154445 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Creating backup snapshot='backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764' {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}} Nov 13 21:42:14.921220 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Performing differential transfer from 'volume-6aa0150e-3650-4ac5-8171-75ce1947688f' to 'volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:42:14.921866 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_kpt4oncd --pool volumes volumes/volume-6aa0150e-3650-4ac5-8171-75ce1947688f@backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:42:14.921866 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:42:59.525959 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Differential backup transfer completed in 44.6043s {{(pid=129022) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}} Nov 13 21:42:59.527471 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Getting metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 13 21:42:59.541370 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=129022) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 13 21:42:59.541914 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Getting metadata type 'volume-metadata' {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 13 21:42:59.549108 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] No metadata type 'volume-metadata' available {{(pid=129022) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 13 21:42:59.549867 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Getting metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 13 21:42:59.557174 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=129022) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 13 21:42:59.557788 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Backing up metadata for volume 6aa0150e-3650-4ac5-8171-75ce1947688f. {{(pid=129022) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}} Nov 13 21:42:59.833518 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Backup 'a621b34b-2c3b-42af-a81a-af3652e012a3' of volume 6aa0150e-3650-4ac5-8171-75ce1947688f finished. {{(pid=129022) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}} Nov 13 21:42:59.835319 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-6aa0150e-3650-4ac5-8171-75ce1947688f', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '6aa0150e-3650-4ac5-8171-75ce1947688f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': True}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:43:00.116588 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] <== disconnect_volume: return (279ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:43:00.304518 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-f87a705e-5549-4097-9c7b-4cda4bc51bf1 tempest-VolumesBackupsTest-1116982884 None] Create backup finished. backup: a621b34b-2c3b-42af-a81a-af3652e012a3. Nov 13 21:43:03.829100 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Restore backup started, backup: a621b34b-2c3b-42af-a81a-af3652e012a3 volume: 179cd4a0-8a38-40e1-8e48-4d0024a54b51. Nov 13 21:43:03.845935 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:43:03.848159 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.856377 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:43:03.856760 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[65500bd1-4923-4615-8ed6-24760cb4987a]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator. The InitiatorName must be unique\n## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:ca9565f7345\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:43:03.858022 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.863910 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[06d51476-3f9a-425c-b7c7-26caaeb93e0a]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:43:03.864474 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:43:03.865742 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[06d51476-3f9a-425c-b7c7-26caaeb93e0a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:43:03.866059 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.872173 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}} Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6b036333-9ade-4223-abb0-41de59af30b2]: [Errno 2] No such file or directory: 'systool' {{(pid=150304) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}} Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: Traceback (most recent call last): Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: ret = func(*f_args, **f_kwargs) Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: return func(*args, **kwargs) Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: return putils.execute(on_execute=on_execute, Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: obj = subprocess.Popen(cmd, Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 858, in __init__ Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 13 21:43:03.873171 ncdv-master-1 cinder-backup[129022]: FileNotFoundError: [Errno 2] No such file or directory: 'systool' Nov 13 21:43:03.874353 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b036333-9ade-4223-abb0-41de59af30b2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:43:03.875320 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.884380 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:43:03.884794 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[c42302dd-d1a6-43cd-a053-b7a5901009a9]: (4, ('/dev/sda1\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:43:03.885905 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.895856 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:43:03.896189 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[43bb1f2f-422a-4266-a01f-f819d5768be4]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:43:03.897335 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.916011 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.017s {{(pid=150304) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:43:03.916011 ncdv-master-1 cinder-backup[129022]: DEBUG oslo.privsep.daemon [-] privsep: reply[f33943ac-8b60-4294-b5a6-6440e808693d]: (4, ('5ffe6c49-c387-ca49-bf33-7dad43decd16\n', '')) {{(pid=150304) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}} Nov 13 21:43:03.916226 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Running cmd (subprocess): nvme version {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Nov 13 21:43:03.934741 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_concurrency.processutils [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=129022) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Nov 13 21:43:03.935803 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.utils [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] <== get_connector_properties: return (89ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:ca9565f7345', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '5ffe6c49-c387-ca49-bf33-7dad43decd16', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:da24d0af-593f-4bab-a9cf-224425f3c726'} {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:43:04.750003 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connector [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Factory for rbd on None {{(pid=129022) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}} Nov 13 21:43:04.751455 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] ==> connect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-179cd4a0-8a38-40e1-8e48-4d0024a54b51', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '179cd4a0-8a38-40e1-8e48-4d0024a54b51', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:43:04.752507 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.linuxrbd [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] opening connection to ceph cluster (timeout=-1). {{(pid=129022) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}} Nov 13 21:43:04.915780 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] <== connect_volume: return (163ms) {'path': } {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:43:04.924001 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:43:04.935549 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Starting restore from Ceph backup=a621b34b-2c3b-42af-a81a-af3652e012a3 to volume=volume-179cd4a0-8a38-40e1-8e48-4d0024a54b51 {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}} Nov 13 21:43:05.015678 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Looking for snapshot of backup base 'volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:43:05.016008 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Found snapshot 'backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:43:05.021767 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Volume file is RBD. {{(pid=129022) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}} Nov 13 21:43:05.071506 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Attempting differential restore. {{(pid=129022) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}} Nov 13 21:43:05.072051 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Attempting incremental restore from base='volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' snap='backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764' {{(pid=129022) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}} Nov 13 21:43:05.072462 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Performing differential transfer from 'volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' to 'volume-179cd4a0-8a38-40e1-8e48-4d0024a54b51' {{(pid=129022) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}} Nov 13 21:43:05.072904 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3@backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764 -' into... {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}} Nov 13 21:43:05.073312 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_4x4adhzy --pool volumes - volumes/volume-179cd4a0-8a38-40e1-8e48-4d0024a54b51' {{(pid=129022) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}} Nov 13 21:43:10.846874 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:43:10.847508 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:43:56.613104 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Restore transfer completed in 51.5403s {{(pid=129022) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}} Nov 13 21:43:56.614196 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}} Nov 13 21:43:56.892471 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Restoring volume base metadata {{(pid=129022) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 13 21:43:56.995176 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] No metadata of type 'volume-metadata' to restore {{(pid=129022) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 13 21:43:56.995553 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.driver [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Restoring volume glance metadata {{(pid=129022) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 13 21:43:58.146607 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Restore to volume 179cd4a0-8a38-40e1-8e48-4d0024a54b51 finished successfully. {{(pid=129022) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}} Nov 13 21:43:58.148120 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] ==> disconnect_volume: call "{'self': , 'connection_properties': {'name': 'volumes/volume-179cd4a0-8a38-40e1-8e48-4d0024a54b51', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '179cd4a0-8a38-40e1-8e48-4d0024a54b51', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': }, 'force': True, 'ignore_errors': False}" {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}} Nov 13 21:43:58.434437 ncdv-master-1 cinder-backup[129022]: DEBUG os_brick.initiator.connectors.rbd [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] <== disconnect_volume: return (285ms) None {{(pid=129022) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}} Nov 13 21:43:58.479568 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.manager [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Encryption key ID for volume 179cd4a0-8a38-40e1-8e48-4d0024a54b51 already matches encryption key ID in backup a621b34b-2c3b-42af-a81a-af3652e012a3. {{(pid=129022) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}} Nov 13 21:43:58.686909 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-b81f370f-44bc-4d11-b450-04c3fce6f09d tempest-VolumesBackupsTest-1116982884 None] Finished restoring backup a621b34b-2c3b-42af-a81a-af3652e012a3 to volume 179cd4a0-8a38-40e1-8e48-4d0024a54b51. Nov 13 21:44:10.842902 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:44:10.843545 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:44:13.338966 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Delete backup started, backup: a621b34b-2c3b-42af-a81a-af3652e012a3. Nov 13 21:44:13.339719 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Delete started for backup=a621b34b-2c3b-42af-a81a-af3652e012a3 {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}} Nov 13 21:44:13.340145 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Trying diff format basename='volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' for backup base image of volume 6aa0150e-3650-4ac5-8171-75ce1947688f. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}} Nov 13 21:44:13.460070 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Looking for snapshot of backup base 'volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}} Nov 13 21:44:13.460481 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Found snapshot 'backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764' {{(pid=129022) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}} Nov 13 21:44:13.460888 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Deleting backup snapshot='backup.a621b34b-2c3b-42af-a81a-af3652e012a3.snap.1636839733.1537764' {{(pid=129022) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}} Nov 13 21:44:16.744430 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Deleting backup base image='volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' of volume 6aa0150e-3650-4ac5-8171-75ce1947688f. Nov 13 21:44:22.837772 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_db.sqlalchemy.engines [None req-c1476bf3-0c1d-412b-986f-86f46a85b647 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129022) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Nov 13 21:44:29.129115 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Base backup image='volume-6aa0150e-3650-4ac5-8171-75ce1947688f.backup.a621b34b-2c3b-42af-a81a-af3652e012a3' of volume 6aa0150e-3650-4ac5-8171-75ce1947688f deleted. {{(pid=129022) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}} Nov 13 21:44:29.446311 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.backup.drivers.ceph [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Delete of backup 'a621b34b-2c3b-42af-a81a-af3652e012a3' for volume '6aa0150e-3650-4ac5-8171-75ce1947688f' finished. {{(pid=129022) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}} Nov 13 21:44:29.572483 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.quota [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Created reservations ['551c9276-75fe-43be-8f9d-bc03fb95da5e', '928481be-2c14-4751-8d30-a2450972f9b9'] {{(pid=129022) reserve /opt/stack/cinder/cinder/quota.py:755}} Nov 13 21:44:29.759455 ncdv-master-1 cinder-backup[129022]: INFO cinder.backup.manager [None req-2250766d-4089-4b40-b306-c9ee69ad150f tempest-VolumesBackupsTest-1116982884 None] Delete backup finished, backup a621b34b-2c3b-42af-a81a-af3652e012a3 deleted. Nov 13 21:45:10.848302 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:45:10.849057 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:46:10.845055 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:46:10.845756 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:47:10.850221 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:47:10.851342 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:48:10.846920 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:48:10.847598 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:49:10.856990 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:49:10.856990 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:50:10.849485 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:50:10.850161 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:51:10.855210 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:51:10.855840 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:52:10.859904 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:52:10.860640 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:53:10.865114 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:53:10.865824 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:54:10.870226 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:54:10.870910 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:55:10.866358 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:55:10.867043 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:56:10.871756 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:56:10.872442 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:57:10.877084 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:57:10.877748 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 21:58:10.882596 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 21:58:10.883227 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:00:10.889378 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:00:10.890046 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:01:10.886290 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:01:10.886961 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:02:10.883260 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:02:10.883908 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:03:10.889320 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:03:10.890030 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:04:10.895372 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:04:10.896033 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:05:10.900005 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:05:10.900678 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:06:10.896980 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:06:10.897709 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:07:10.901920 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:07:10.902596 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:08:10.906532 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:08:10.907168 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:09:10.911457 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:09:10.912132 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:10:10.915821 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:10:10.955806 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:11:10.912340 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:11:10.913128 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:12:10.917662 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:12:10.918400 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:13:10.914813 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:13:10.915499 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:14:10.910644 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:14:10.911312 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:15:10.915546 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:15:10.916211 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:16:10.920720 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:16:10.921420 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:17:10.916726 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:17:10.917411 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:18:10.921827 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:18:10.922492 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:19:10.927586 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:19:10.928218 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:20:10.933678 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:20:10.934232 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:21:10.938525 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:21:10.939061 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:22:10.943203 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:22:10.943758 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:23:10.948265 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:23:10.949433 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:24:10.946920 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:24:10.947478 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:25:10.952331 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:25:10.952958 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:26:10.948650 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:26:10.949251 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:27:10.953816 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:27:10.954376 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:28:10.959060 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:28:10.959772 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:29:10.963852 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:29:10.964515 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:30:10.968760 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:30:10.969440 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:31:10.974051 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:31:10.974697 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:32:10.979043 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:32:10.979744 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:33:10.984298 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:33:10.984999 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:34:10.989645 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:34:10.994365 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:35:10.985856 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:35:10.987007 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:36:10.992262 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:36:10.992935 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:37:10.997638 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:37:10.998290 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:38:10.994183 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:38:10.994856 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:39:10.989921 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:39:10.990579 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:40:10.995278 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:40:10.995954 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:41:11.000232 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:41:11.000932 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 13 22:42:11.005293 ncdv-master-1 cinder-backup[129022]: DEBUG oslo_service.periodic_task [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=129022) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Nov 13 22:42:11.005922 ncdv-master-1 cinder-backup[129022]: DEBUG cinder.manager [None req-eaa27a43-1413-49bc-aa8e-60b9300cb5f3 None None] Notifying Schedulers of capabilities ... {{(pid=129022) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}