-- Logs begin at Sat 2021-12-11 20:01:09 UTC, end at Sat 2021-12-11 22:14:52 UTC. --
Dec 11 20:38:53.491669 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service.
Dec 11 20:38:56.711554 ncdv-master-1 cinder-backup[133643]: INFO cinder.cmd.backup [-] Backup running with 1 processes.
Dec 11 20:38:56.713468 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=133643) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}}
Dec 11 20:38:56.714030 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=133643) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}}
Dec 11 20:38:56.884442 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_db.sqlalchemy.engines [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133643) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 20:38:57.126689 ncdv-master-1 cinder-backup[133643]: INFO cinder.rpc [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version.
Dec 11 20:38:57.130905 ncdv-master-1 cinder-backup[133643]: INFO cinder.rpc [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version.
Dec 11 20:38:57.135124 ncdv-master-1 cinder-backup[133643]: INFO cinder.rpc [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version.
Dec 11 20:38:57.139136 ncdv-master-1 cinder-backup[133643]: INFO cinder.rpc [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version.
Dec 11 20:38:57.143860 ncdv-master-1 cinder-backup[133643]: INFO cinder.rpc [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
Dec 11 20:38:57.148120 ncdv-master-1 cinder-backup[133643]: INFO cinder.rpc [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version.
Dec 11 20:38:57.197899 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_db.sqlalchemy.engines [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133643) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 20:38:57.225836 ncdv-master-1 cinder-backup[133643]: INFO oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Starting 1 workers
Dec 11 20:38:57.231805 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Started child 134086 {{(pid=133643) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Dec 11 20:38:57.233956 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Full set of CONF: {{(pid=133643) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}}
Dec 11 20:38:57.234578 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] ******************************************************************************** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}}
Dec 11 20:38:57.235268 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] Configuration options gathered from: {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}}
Dec 11 20:38:57.235929 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}}
Dec 11 20:38:57.236769 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}}
Dec 11 20:38:57.237210 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] ================================================================================ {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}}
Dec 11 20:38:57.238299 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] allow_availability_zone_fallback = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.238901 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] allow_compression_on_image_upload = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.239628 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] allowed_direct_url_schemes     = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.240966 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] api_rate_limit                 = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.241598 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] auth_strategy                  = keystone {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.242137 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_availability_zone      = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.242750 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_api_class               = cinder.backup.api.API {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.243455 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_chunk_size         = 134217728 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.244170 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_conf               = /etc/ceph/ceph.conf {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.244708 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_image_journals     = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.245399 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_pool               = backups {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.245933 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_stripe_count       = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.246550 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_stripe_unit        = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.247082 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_ceph_user               = cinder-bak {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.247825 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_driver                  = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.248381 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_driver_init_check_interval = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.248920 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_driver_stats_polling_interval = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.249495 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_manager                 = cinder.backup.manager.BackupManager {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.250041 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_max_operations          = 15 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.250999 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_metadata_version        = 2 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.251672 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_name_template           = backup-%s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.252600 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_native_threads_pool_size = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.253465 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_object_number_per_notification = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.254453 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_service_inithost_offload = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.255026 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_timer_interval          = 120 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.255726 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_use_same_host           = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.255965 ncdv-master-1 cinder-backup[134086]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0)
Dec 11 20:38:57.256838 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_use_temp_snapshot       = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.257383 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backup_workers                 = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.258051 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] chap_password                  = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.258686 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] chap_username                  =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.259254 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.259912 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_internal_tenant_project_id = 0fa26583c3854499bf9a64bce9b104c1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.261188 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_internal_tenant_user_id = 206c81b5fd6548d9894b69b752029642 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.261736 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] compression_format             = gzip {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.262312 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] config_dir                     = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.262812 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] config_file                    = ['/etc/cinder/cinder.conf'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.263372 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] config_source                  = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.264025 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] consistencygroup_api_class     = cinder.consistencygroup.api.API {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.264609 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] control_exchange               = openstack {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.265198 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] debug                          = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.265817 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] default_availability_zone      = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.266409 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] default_group_type             = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.267340 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.268248 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] default_volume_type            = ceph {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.268858 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] driver_client_cert             = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.269357 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] driver_client_cert_key         = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.269883 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] driver_data_namespace          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.270387 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] driver_ssl_cert_path           = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.271704 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] driver_ssl_cert_verify         = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.272248 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] driver_use_ssl                 = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.272719 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] enable_new_services            = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.273259 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] enable_unsupported_driver      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.273825 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] enabled_backends               = ['ceph'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.274433 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] enforce_multipath_for_image_xfer = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.275055 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] filter_function                = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.275694 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_api_insecure            = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.276261 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_api_servers             = ['http://10.222.0.18/image'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.276781 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_api_ssl_compression     = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.277300 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_ca_certificates_file    = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.277843 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_catalog_info            = image:glance:publicURL {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.278451 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_certfile                = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.279150 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.279685 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_keyfile                 = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.280199 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_num_retries             = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.280694 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] glance_request_timeout         = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.281202 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] goodness_function              = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.281930 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] graceful_shutdown_timeout      = 5 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.282456 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] group_api_class                = cinder.group.api.API {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.283069 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] host                           = ncdv-master-1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.283601 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iet_conf                       = /etc/iet/ietd.conf {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.284309 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_compress_on_upload       = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.284816 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_conversion_address_space_limit = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.285353 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_conversion_cpu_limit     = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.286143 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_conversion_dir           = /opt/stack/data/cinder/conversion {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.287247 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_upload_use_cinder_backend = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.287762 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_upload_use_internal_tenant = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.288386 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_volume_cache_enabled     = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.288905 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_volume_cache_max_count   = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.289672 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] image_volume_cache_max_size_gb = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.290186 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] instance_format                = [instance: %(uuid)s]  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.290754 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.291310 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iscsi_iotype                   = fileio {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.292293 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iscsi_secondary_ip_addresses   = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.292829 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iscsi_target_flags             =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.293498 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iscsi_write_cache              = on {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.294014 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iser_helper                    = tgtadm {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.294915 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iser_ip_address                = 10.222.0.18 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.295500 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iser_port                      = 3260 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.296152 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] iser_target_prefix             = iqn.2010-10.org.openstack: {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.296663 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_config_append              = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.297182 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.297702 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_dir                        = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.298201 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_file                       = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.298767 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_options                    = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.299346 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_rotate_interval            = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.299919 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_rotate_interval_type       = days {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.300505 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] log_rotation_type              = none {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.300962 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.301432 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.302320 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.302908 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.303363 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.303897 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] max_age                        = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.304540 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] max_logfile_count              = 30 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.305248 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] max_logfile_size_mb            = 200 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.305908 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] max_over_subscription_ratio    = 20.0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.306463 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] message_reap_interval          = 86400 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.307039 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] message_ttl                    = 2592000 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.307560 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] monkey_patch                   = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.308121 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] monkey_patch_modules           = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.308694 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] my_ip                          = 10.222.0.18 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.309228 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] no_snapshot_gb_quota           = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.309792 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] num_iser_scan_tries            = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.310327 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] num_shell_tries                = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.310855 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] num_volume_device_scan_tries   = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.311392 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] nvmet_ns_id                    = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.312088 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] nvmet_port_id                  = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.312789 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_max_limit                = 1000 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.313323 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_volume_ext_list          = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.316815 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.317434 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_volume_listen            = 0.0.0.0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.318128 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_volume_listen_port       = 8776 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.318886 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_volume_use_ssl           = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.319497 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] osapi_volume_workers           = 2 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.320037 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] per_volume_size_limit          = -1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.321370 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] periodic_fuzzy_delay           = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.321907 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] periodic_interval              = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.322653 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] public_endpoint                = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.323215 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] publish_errors                 = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.323796 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_backup_gigabytes         = 1000 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.324327 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_backups                  = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.325028 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_consistencygroups        = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.325605 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_driver                   = cinder.quota.DbQuotaDriver {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.326119 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_gigabytes                = 1000 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.326813 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_groups                   = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.327370 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_snapshots                = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.327940 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] quota_volumes                  = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.328487 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] rate_limit_burst               = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.329130 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] rate_limit_except_level        = CRITICAL {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.329690 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] rate_limit_interval            = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.329934 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.lockutils [None req-1b01de52-a124-4919-8734-375ba638f75f None None] Lock "cleanup_incomplete_backups_133643" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.002s {{(pid=134086) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}}
Dec 11 20:38:57.330374 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] replication_device             = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.330512 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-1b01de52-a124-4919-8734-375ba638f75f None None] Cleaning up incomplete backup operations.
Dec 11 20:38:57.331033 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] report_discard_supported       = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.331553 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] report_interval                = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.332157 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] reservation_clean_interval     = 86400 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.332782 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] reservation_expire             = 86400 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.333304 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] reserved_percentage            = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.334421 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] resource_query_filters_file    = /etc/cinder/resource_filters.json {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.335152 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] restore_discard_excess_bytes   = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.335862 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] rootwrap_config                = /etc/cinder/rootwrap.conf {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.336603 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] rpc_response_timeout           = 120 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.337120 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] run_external_periodic_tasks    = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.337641 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] scheduler_manager              = cinder.scheduler.manager.SchedulerManager {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.338274 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] scst_target_driver             = iscsi {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.338823 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] scst_target_iqn_name           = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.339374 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_down_time              = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.339966 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] snapshot_name_template         = snapshot-%s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.340523 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] split_loggers                  = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.340978 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] state_path                     = /opt/stack/data/cinder {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.341492 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] storage_availability_zone      = nova {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.342019 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] storage_protocol               = iscsi {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.343051 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] syslog_log_facility            = LOG_USER {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.343591 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] target_helper                  = lioadm {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.344285 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] target_ip_address              = 10.222.0.18 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.344830 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] target_port                    = 3260 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.345967 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] target_prefix                  = iqn.2010-10.org.openstack: {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.346647 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] target_protocol                = iscsi {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.347199 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] trace_flags                    = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.347725 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] transfer_api_class             = cinder.transfer.api.API {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.348258 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] transport_url                  = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.348814 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] until_refresh                  = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.349968 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_chap_auth                  = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.350647 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_default_quota_class        = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.351165 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_eventlog                   = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.351696 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_journal                    = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.352248 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_json                       = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.352955 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_multipath_for_image_xfer   = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.353460 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_stderr                     = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.353961 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] use_syslog                     = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.354606 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] verify_glance_signatures       = enabled {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.355135 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_api_class               = cinder.volume.api.API {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.355759 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_backend_name            = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.356305 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_clear                   = zero {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.356925 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_clear_ionice            = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.357427 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_clear_size              = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.357955 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_copy_blkio_cgroup_name  = cinder-volume-copy {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.358466 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_copy_bps_limit          = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.359014 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_dd_blocksize            = 1M {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.359649 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_manager                 = cinder.volume.manager.VolumeManager {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.360219 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_name_template           = volume-%s {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.360813 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volume_usage_audit_period      = month {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.361430 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] volumes_dir                    = /opt/stack/data/cinder/volumes {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.361940 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] watch_log_file                 = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Dec 11 20:38:57.362656 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.363400 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.363975 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.enforce_new_defaults = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.364489 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.enforce_scope      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.365087 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.policy_default_rule = default {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.365783 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.366346 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.policy_file        = policy.yaml {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.367025 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.367560 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.368124 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.368646 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.369181 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.369819 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_concurrency.disable_process_locking = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.370441 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_concurrency.lock_path     = /opt/stack/data/cinder {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.371204 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.admin_password = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.371756 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.372261 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.admin_token = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.372769 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.admin_user  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.373499 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_admin_prefix =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.374035 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_host   = 127.0.0.1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.374596 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_port   = 35357 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.375140 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_protocol = https {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.375828 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_section = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.376459 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_type   = password {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.377205 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_uri    = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.377717 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.auth_version = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.378217 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.cache       = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.378837 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.cafile      = /opt/stack/data/ca-bundle.pem {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.379375 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.certfile    = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.380021 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.delay_auth_decision = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.380598 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.381096 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.http_connect_timeout = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.381602 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.382099 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.identity_uri = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.382640 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.include_service_catalog = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.383415 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.insecure    = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.383943 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.interface   = public {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.384454 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.keyfile     = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.385014 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.385573 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.386214 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.386756 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.387390 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.387894 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.388447 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.388968 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.389544 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.390424 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.region_name = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.391011 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.391523 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.service_token_roles_required = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.392040 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.service_type = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.392590 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.token_cache_time = 300 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.393492 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.394152 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.backend                  = dogpile.cache.null {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.394724 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.backend_argument         = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.395365 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.config_prefix            = cache.oslo {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.396534 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.debug_cache_backend      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.397117 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.enabled                  = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.397760 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.expiration_time          = 600 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.398276 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_dead_retry      = 300 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.398833 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.399389 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.400014 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_pool_maxsize    = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.400514 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.401065 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.401622 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.402152 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.proxies                  = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.402681 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.tls_allowed_ciphers      = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.403204 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.tls_cafile               = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.403880 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.tls_certfile             = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.404377 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.tls_enabled              = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.404877 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cache.tls_keyfile              = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.405416 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] coordination.backend_url       = etcd3+http://10.222.0.18:2379 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.405966 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.backend               = sqlalchemy {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.406420 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.connection            = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.407029 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.connection_debug      = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.407634 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.connection_parameters =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.408144 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.connection_recycle_time = 3600 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.408811 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.connection_trace      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.409364 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.db_inc_retry_interval = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.409864 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.db_max_retries        = 20 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.410459 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.db_max_retry_interval = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.411141 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.db_retry_interval     = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.412389 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.max_overflow          = 50 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.413449 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.max_pool_size         = 5 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.414546 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.max_retries           = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.415093 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.mysql_enable_ndb      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.415624 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.416141 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.pool_timeout          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.416649 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.retry_interval        = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.417217 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.slave_connection      = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.417814 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.sqlite_synchronous    = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.418307 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] database.use_db_reconnect      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.418885 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.419376 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.419877 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.420480 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.421061 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.421633 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.connection_string     = messaging:// {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.422079 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.enabled               = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.422607 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.es_doc_type           = notification {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.423225 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.es_scroll_size        = 10000 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.423882 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.es_scroll_time        = 2m {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.424440 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.filter_error_trace    = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.424950 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.425515 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.sentinel_service_name = mymaster {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.426011 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.socket_timeout        = 0.1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.426533 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] profiler.trace_sqlalchemy      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.427233 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] privsep_osbrick.capabilities   = [21] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.427865 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] privsep_osbrick.group          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.428364 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] privsep_osbrick.helper_command = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.428867 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] privsep_osbrick.logger_name    = oslo_privsep.daemon {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.429387 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.429896 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] privsep_osbrick.user           = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.430564 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_sys_admin.capabilities  = [0, 1, 2, 3, 12, 21] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.431052 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_sys_admin.group         = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.431535 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_sys_admin.helper_command = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.432044 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_sys_admin.logger_name   = oslo_privsep.daemon {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.432537 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.433038 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] cinder_sys_admin.user          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.433560 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.auth_section      = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.434215 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.auth_type         = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.434750 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.cafile            = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.435238 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.certfile          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.435759 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.collect_timing    = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.436593 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.insecure          = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.437412 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.keyfile           = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.438409 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.send_service_user_token = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.439155 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.split_loggers     = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.439879 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] service_user.timeout           = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.440848 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.backend_availability_zone = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.441555 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.chap_password = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.442121 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.chap_username =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.442676 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.443248 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.443783 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.444424 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.driver_client_cert = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.444912 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.driver_client_cert_key = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.445432 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.driver_data_namespace = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.445933 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.446461 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.447219 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.driver_use_ssl = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.449876 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.enable_deferred_deletion = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.450367 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.enable_unsupported_driver = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.450880 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.451352 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.filter_function = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.455249 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.goodness_function = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.455777 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iet_conf      = /etc/iet/ietd.conf {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.456264 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.456745 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.457222 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.457715 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.458186 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.459204 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iscsi_iotype  = fileio {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.464138 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.464800 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iscsi_target_flags =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.465335 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iscsi_write_cache = on {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.465871 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iser_helper   = tgtadm {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.466624 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iser_ip_address = 10.222.0.18 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.467309 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iser_port     = 3260 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.468277 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.468796 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.469312 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.469906 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.num_shell_tries = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.470400 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.471023 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.nvmet_ns_id   = 10 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.471587 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.nvmet_port_id = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.472177 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.472830 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rados_connection_interval = 5 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.473328 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rados_connection_retries = 3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.473978 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_ceph_conf =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.475436 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.475948 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.476609 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.477103 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.477704 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_pool      = rbd {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.478305 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_secret_uuid = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.478820 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.479327 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.rbd_user      = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.479923 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.480412 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.replication_device = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.481018 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.report_discard_supported = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.481519 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.482022 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.reserved_percentage = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.482547 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.scst_target_driver = iscsi {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.483142 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.scst_target_iqn_name = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.483662 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.storage_protocol = iscsi {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.484571 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.target_helper = tgtadm {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.485366 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.target_ip_address = 10.222.0.18 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.485878 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.target_port   = 3260 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.486584 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.487096 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.target_protocol = iscsi {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.487604 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.trace_flags   = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.488186 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.unique_fqdn_network = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.488780 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.use_chap_auth = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.489275 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.489852 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_backend_name = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.490489 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_clear  = zero {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.491312 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_clear_ionice = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.491830 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_clear_size = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.492341 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.492841 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.493447 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.494051 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] backend_defaults.volumes_dir   = /opt/stack/data/cinder/volumes {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.494746 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_reports.file_event_handler = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.495366 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.495879 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_reports.log_dir           = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.496537 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.497052 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.497569 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.498180 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.498812 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.499307 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.499834 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.500426 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.500930 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.501539 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.502076 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.502615 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.503216 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.503735 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.504224 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.504894 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.505516 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.506016 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.506608 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.507135 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.507680 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.508940 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.ssl      = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.509466 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.509986 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.510639 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.511147 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.511826 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_notifications.driver = [] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.512356 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_notifications.retry = -1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.513125 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.513732 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] oslo_messaging_notifications.transport_url = **** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.514308 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] key_manager.backend            = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.515003 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.515605 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.barbican_api_version  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.516095 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.barbican_endpoint     = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.516745 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.barbican_endpoint_type = public {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.517248 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.barbican_region_name  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.517735 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.cafile                = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.518217 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.certfile              = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.518955 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.collect_timing        = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.519460 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.insecure              = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.519964 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.keyfile               = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.520786 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.number_of_retries     = 60 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.522420 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.retry_delay           = 1 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.522957 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.send_service_user_token = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.523598 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.split_loggers         = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.524101 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.timeout               = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.524605 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.verify_ssl            = True {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.525198 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican.verify_ssl_path       = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.525793 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.auth_section = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.526301 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.auth_type = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.526938 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.cafile   = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.527450 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.certfile = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.527943 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.collect_timing = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.528437 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.insecure = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.529144 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.keyfile  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.529637 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.split_loggers = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.530139 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] barbican_service_user.timeout  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.531352 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.approle_role_id          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.531882 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.approle_secret_id        = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.532495 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.cafile                   = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.532984 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.certfile                 = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.533489 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.collect_timing           = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.534703 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.insecure                 = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.535548 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.keyfile                  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.536226 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.kv_mountpoint            = secret {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.537727 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.kv_version               = 2 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.538246 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.namespace                = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.539120 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.root_token_id            = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.539730 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.split_loggers            = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.540234 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.ssl_ca_crt_file          = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.540814 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.timeout                  = None {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.541617 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.use_ssl                  = False {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.542142 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Dec 11 20:38:57.542760 ncdv-master-1 cinder-backup[133643]: DEBUG oslo_service.service [None req-c6087945-8cea-4fcc-81f0-fce7472f9678 None None] ******************************************************************************** {{(pid=133643) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}}
Dec 11 20:38:57.606879 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-1b01de52-a124-4919-8734-375ba638f75f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 20:38:57.643889 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.lockutils [None req-1b01de52-a124-4919-8734-375ba638f75f None None] Lock "cleanup_incomplete_backups_133643" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.315s {{(pid=134086) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}}
Dec 11 20:38:57.651007 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-1b01de52-a124-4919-8734-375ba638f75f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 20:38:57.658606 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-1b01de52-a124-4919-8734-375ba638f75f None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:38:57.721170 ncdv-master-1 cinder-backup[134086]: INFO cinder.keymgr.migration [None req-0a317c91-8fda-411b-af05-bf8c1a6a3ab0 None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Dec 11 20:38:57.729040 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.service [None req-1b01de52-a124-4919-8734-375ba638f75f None None] Creating RPC server for service cinder-backup {{(pid=134086) start /opt/stack/cinder/cinder/service.py:231}}
Dec 11 20:38:57.743670 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.service [None req-1b01de52-a124-4919-8734-375ba638f75f None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=134086) start /opt/stack/cinder/cinder/service.py:237}}
Dec 11 20:40:45.785972 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:40:45.786728 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:41:45.781629 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:41:45.782323 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:42:45.778097 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:42:45.778834 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:43:45.782822 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:43:45.783494 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:44:45.788767 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:44:45.789434 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:45:45.794031 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:45:45.794716 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:46:45.799354 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:46:45.800201 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:47:45.804312 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:47:45.805023 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:48:45.803304 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:48:45.804322 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:49:45.807347 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:49:45.807935 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:50:45.806230 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:50:45.806824 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:51:45.807188 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:51:45.808447 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:52:45.807217 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:52:45.807813 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:53:45.811999 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:53:45.812742 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:54:45.812756 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:54:45.813347 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:55:45.817825 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:55:45.818522 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:56:45.822964 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:56:45.823622 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:57:45.821612 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:57:45.822265 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:58:45.827566 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:58:45.828650 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 20:59:45.830993 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 20:59:45.831998 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:00:45.829110 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:00:45.829698 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:01:45.834828 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:01:45.835517 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:02:45.840350 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:02:45.840966 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:03:45.838678 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:03:45.839301 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:03:57.194907 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Create backup started, backup: 750f3b0c-4514-4200-82f6-5f96dd472cdc volume: 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.
Dec 11 21:03:57.210367 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:03:57.212381 ncdv-master-1 cinder-backup[134086]: INFO oslo.privsep.daemon [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpk3kw23mw/privsep.sock']
Dec 11 21:03:57.301919 ncdv-master-1 sudo[158094]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpk3kw23mw/privsep.sock
Dec 11 21:03:57.302835 ncdv-master-1 sudo[158094]: pam_unix(sudo:session): session opened for user root by (uid=0)
Dec 11 21:03:57.991439 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-2c9f83c9-e9b4-413e-978e-77caa8037e44 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:03:59.410636 ncdv-master-1 sudo[158094]: pam_unix(sudo:session): session closed for user root
Dec 11 21:03:59.415177 ncdv-master-1 cinder-backup[134086]: INFO oslo.privsep.daemon [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Spawned new privsep daemon via rootwrap
Dec 11 21:03:59.416379 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Accepted privsep connection to /tmp/tmpk3kw23mw/privsep.sock {{(pid=134086) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}}
Dec 11 21:03:59.418780 ncdv-master-1 cinder-backup[134086]: INFO oslo.privsep.daemon [-] privsep daemon starting
Dec 11 21:03:59.419330 ncdv-master-1 cinder-backup[134086]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Dec 11 21:03:59.419749 ncdv-master-1 cinder-backup[134086]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Dec 11 21:03:59.420271 ncdv-master-1 cinder-backup[134086]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 158115
Dec 11 21:03:59.431347 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[68fa37a3-2415-4405-bb2f-aebadfd4a7f1]: (2,) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.062353 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.086057 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.023s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:00.087295 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[21d79181-e835-47a8-b619-8c75fa5b436b]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.090459 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.112185 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a3773164-3ea0-47ec-ab6d-c8696cdec1bd]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:04:00.132208 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:04:00.133848 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3773164-3ea0-47ec-ab6d-c8696cdec1bd]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.135440 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.162073 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a5473340-2e6a-4383-9f71-32a515e43752]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:04:00.162371 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:04:00.163759 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5473340-2e6a-4383-9f71-32a515e43752]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.165280 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.225309 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.060s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:00.226359 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4b34c04-e9d0-4707-afdc-81e87d88a2cb]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.228146 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.333552 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.103s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:00.335079 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[eea2ad75-e680-4d4a-b205-2e085a17693f]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.338404 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.359904 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.022s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:00.360565 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f79c2b17-a4a9-490c-b912-fd06fcd83600]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:00.361773 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:00.424614 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] CMD "nvme version" returned: 0 in 0.063s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:00.433452 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] <== get_connector_properties: return (3221ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:04:01.815967 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:04:01.815967 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2669c8e0>, 'connection_properties': {'name': 'volumes/volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:04:01.817503 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:04:01.944234 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] <== connect_volume: return (127ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c266f50a0>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:04:01.982870 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Starting backup of volume='7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:04:01.984223 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Volume file is RBD: attempting optimized backup {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Dec 11 21:04:02.061742 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Creating base image 'volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' {{(pid=134086) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Dec 11 21:04:02.062832 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:04:03.798052 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Using --from-snap 'None' for incremental backup of volume 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d. {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Dec 11 21:04:03.798712 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Creating backup snapshot='backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025' {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Dec 11 21:04:04.770157 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Performing differential transfer from 'volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d' to 'volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:04:04.770592 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd__ksz_cwc --pool volumes volumes/volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d@backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:04:04.771256 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:04:13.898852 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Differential backup transfer completed in 9.1279s {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Dec 11 21:04:13.900933 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:04:13.929136 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:04:13.932733 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:04:13.947728 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:04:13.949024 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:04:13.964646 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Dec 11 21:04:13.967191 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Backing up metadata for volume 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:04:14.060292 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Backup '750f3b0c-4514-4200-82f6-5f96dd472cdc' of volume 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:04:14.062210 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2669c8e0>, 'connection_properties': {'name': 'volumes/volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c266f50a0>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:04:14.116718 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] <== disconnect_volume: return (52ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:04:14.309274 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-c7fbea7c-0edb-4326-b847-f101d51707a3 tempest-TestVolumeBackupRestore-1162231384 None] Create backup finished. backup: 750f3b0c-4514-4200-82f6-5f96dd472cdc.
Dec 11 21:04:16.491806 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Restore backup started, backup: 750f3b0c-4514-4200-82f6-5f96dd472cdc volume: 1db74c9b-76be-4978-a8ac-3b6cb127fc26.
Dec 11 21:04:16.598282 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:04:16.600253 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.652251 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.051s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:16.652800 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fbb4d52-a244-405c-ad03-dc6b56d7a0bc]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:16.654888 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.678276 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0f32b96a-e254-4893-b854-229e247b86bb]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:04:16.679240 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:04:16.680643 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f32b96a-e254-4893-b854-229e247b86bb]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:16.682418 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.695857 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[98e5e9bf-5bca-4adc-9352-9dae632f87c9]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:04:16.696181 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:04:16.697463 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[98e5e9bf-5bca-4adc-9352-9dae632f87c9]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:16.703255 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.727107 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.028s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:16.727387 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[81a9f295-7ea1-4104-b7d1-6eaa6f119f3b]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:16.742972 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.771119 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.041s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:16.771459 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[71fa5bff-5d36-4bde-8c9b-3e3bfb56ca6e]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:16.780794 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.793964 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.014s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:16.793964 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2f739dd-68ab-4f3c-80b3-a909b2948a6d]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:04:16.795056 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:04:16.832657 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] CMD "nvme version" returned: 0 in 0.035s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:04:16.832657 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] <== get_connector_properties: return (232ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:04:17.901593 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:04:17.903073 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2688a100>, 'connection_properties': {'name': 'volumes/volume-1db74c9b-76be-4978-a8ac-3b6cb127fc26', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '1db74c9b-76be-4978-a8ac-3b6cb127fc26', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:04:17.904136 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:04:17.982540 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] <== connect_volume: return (77ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c26622370>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:04:17.996811 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:04:18.028348 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Starting restore from Ceph backup=750f3b0c-4514-4200-82f6-5f96dd472cdc to volume=volume-1db74c9b-76be-4978-a8ac-3b6cb127fc26 {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Dec 11 21:04:18.175877 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Looking for snapshot of backup base 'volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:04:18.176376 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Found snapshot 'backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:04:18.193153 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Volume file is RBD. {{(pid=134086) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Dec 11 21:04:18.259292 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Attempting differential restore. {{(pid=134086) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Dec 11 21:04:18.259744 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Attempting incremental restore from base='volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' snap='backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025' {{(pid=134086) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Dec 11 21:04:18.260316 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Performing differential transfer from 'volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' to 'volume-1db74c9b-76be-4978-a8ac-3b6cb127fc26' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:04:18.260756 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc@backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:04:18.261179 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_3bmxwx8i --pool volumes - volumes/volume-1db74c9b-76be-4978-a8ac-3b6cb127fc26' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:04:23.904793 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Restore transfer completed in 5.6432s {{(pid=134086) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Dec 11 21:04:23.906858 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Dec 11 21:04:23.953372 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Restoring volume base metadata {{(pid=134086) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Dec 11 21:04:23.995277 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] No metadata of type 'volume-metadata' to restore {{(pid=134086) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Dec 11 21:04:23.998507 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Restoring volume glance metadata {{(pid=134086) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Dec 11 21:04:24.251816 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Restore to volume 1db74c9b-76be-4978-a8ac-3b6cb127fc26 finished successfully. {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Dec 11 21:04:24.254744 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2688a100>, 'connection_properties': {'name': 'volumes/volume-1db74c9b-76be-4978-a8ac-3b6cb127fc26', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '1db74c9b-76be-4978-a8ac-3b6cb127fc26', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c26622370>}, 'force': True, 'ignore_errors': False}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:04:24.269597 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] <== disconnect_volume: return (13ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:04:24.327692 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.manager [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Encryption key ID for volume 1db74c9b-76be-4978-a8ac-3b6cb127fc26 already matches encryption key ID in backup 750f3b0c-4514-4200-82f6-5f96dd472cdc. {{(pid=134086) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Dec 11 21:04:24.364500 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-b73ee75c-e34a-411b-844d-f5c0cf3df303 tempest-TestVolumeBackupRestore-1162231384 None] Finished restoring backup 750f3b0c-4514-4200-82f6-5f96dd472cdc to volume 1db74c9b-76be-4978-a8ac-3b6cb127fc26.
Dec 11 21:04:45.845260 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:04:45.845906 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:05:14.752526 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Create backup started, backup: ad504e7d-6609-46da-9e6d-d50578f9abe2 volume: eadf804a-f17e-4ea4-b992-c925e6e57a8e.
Dec 11 21:05:14.779295 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:14.787011 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.808150 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.026s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:14.808711 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d84c43e4-a932-4f5d-a99a-b14109db7268]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:14.813923 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.829542 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[afd8fe5b-fbf8-47a9-baec-2a14cfa284ad]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:05:14.834257 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:05:14.843498 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[afd8fe5b-fbf8-47a9-baec-2a14cfa284ad]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:14.846518 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.871894 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c2afa9ac-734f-4670-9d59-b00da5f32c60]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:05:14.872509 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:05:14.874437 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2afa9ac-734f-4670-9d59-b00da5f32c60]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:14.879203 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.910316 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.033s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:14.910992 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccb1a193-96ce-4e77-a5ff-abfd2ab7ad42]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:14.915010 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.931602 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.018s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:14.931891 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a27877d3-11a0-4df5-a3e2-04dd0614756d]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:14.936054 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.951042 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.017s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:14.951692 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1be9fea2-b253-49e9-9310-09207a1138b8]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:14.952697 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:14.998759 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] CMD "nvme version" returned: 0 in 0.045s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:15.004238 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] <== get_connector_properties: return (223ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:16.233819 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:05:16.236151 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c27016460>, 'connection_properties': {'name': 'volumes/volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'eadf804a-f17e-4ea4-b992-c925e6e57a8e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:16.244057 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:05:16.358086 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] <== connect_volume: return (120ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c26650d30>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:16.365809 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:16.388318 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Starting backup of volume='eadf804a-f17e-4ea4-b992-c925e6e57a8e'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:05:16.390898 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Volume file is RBD: attempting optimized backup {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Dec 11 21:05:16.438632 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Creating base image 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' {{(pid=134086) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Dec 11 21:05:16.439332 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:05:16.515149 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Using --from-snap 'None' for incremental backup of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e. {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Dec 11 21:05:16.517449 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Creating backup snapshot='backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625' {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Dec 11 21:05:17.518038 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Performing differential transfer from 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e' to 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:05:17.519382 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_mmx8j_yj --pool volumes volumes/volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e@backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:05:17.519806 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:05:18.718235 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Differential backup transfer completed in 1.1998s {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Dec 11 21:05:18.721454 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:05:18.738512 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:05:18.744203 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:05:18.804564 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:05:18.804979 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:05:18.824967 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] No metadata type 'volume-glance-metadata' available {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Dec 11 21:05:18.825998 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Backing up metadata for volume eadf804a-f17e-4ea4-b992-c925e6e57a8e. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:05:18.918182 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Backup 'ad504e7d-6609-46da-9e6d-d50578f9abe2' of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:05:18.920284 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c27016460>, 'connection_properties': {'name': 'volumes/volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'eadf804a-f17e-4ea4-b992-c925e6e57a8e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c26650d30>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:18.984191 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] <== disconnect_volume: return (60ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:19.111940 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-42b6b6d5-3e80-4ea5-bb32-bab57e9993db tempest-VolumesBackupsAdminTest-953270115 None] Create backup finished. backup: ad504e7d-6609-46da-9e6d-d50578f9abe2.
Dec 11 21:05:20.161392 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-f3abdf04-8e0b-4a1c-a238-f8a8aa1538e2 tempest-VolumesBackupsAdminTest-1846777574 None] Export record started, backup: ad504e7d-6609-46da-9e6d-d50578f9abe2.
Dec 11 21:05:20.164124 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-f3abdf04-8e0b-4a1c-a238-f8a8aa1538e2 tempest-VolumesBackupsAdminTest-1846777574 None] Export record finished, backup ad504e7d-6609-46da-9e6d-d50578f9abe2 exported.
Dec 11 21:05:20.545070 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-bc3219f6-a390-4e1a-9a68-b11f203904fa tempest-VolumesBackupsAdminTest-1846777574 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiNTA3YjdjM2MtMTEwMi00ZThmLThjMjUtMzBmOWQzZGEyMzhjIiwgInVzZXJfaWQiOiAiMmYyMjRlNGU3YzhhNDJhMmE5YWJiYWMwNTEyMDExM2YiLCAicHJvamVjdF9pZCI6ICIzMDEwMWEwM2RmOGM0ZDE1OTliOWYwMzBlNDI2M2I4YyIsICJ2b2x1bWVfaWQiOiAiZWFkZjgwNGEtZjE3ZS00ZWE0LWI5OTItYzkyNWU2ZTU3YThlIiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC0xNTc5MzE5MzYzIiwgImRpc3BsYXlfZGVzY3JpcHRpb24iOiBudWxsLCAic2VydmljZV9tZXRhZGF0YSI6ICJ7XCJiYXNlXCI6IFwidm9sdW1lLWVhZGY4MDRhLWYxN2UtNGVhNC1iOTkyLWM5MjVlNmU1N2E4ZS5iYWNrdXAuYWQ1MDRlN2QtNjYwOS00NmRhLTllNmQtZDUwNTc4ZjlhYmUyXCJ9IiwgInNlcnZpY2UiOiAiY2luZGVyLmJhY2t1cC5kcml2ZXJzLmNlcGguQ2VwaEJhY2t1cERyaXZlciIsICJvYmplY3RfY291bnQiOiAwLCAidGVtcF92b2x1bWVfaWQiOiBudWxsLCAidGVtcF9zbmFwc2hvdF9pZCI6IG51bGwsICJudW1fZGVwZW5kZW50X2JhY2t1cHMiOiAwLCAic25hcHNob3RfaWQiOiBudWxsLCAiZGF0YV90aW1lc3RhbXAiOiAiMjAyMS0xMi0xMVQyMTowNToxNVoiLCAicmVzdG9yZV92b2x1bWVfaWQiOiBudWxsLCAibWV0YWRhdGEiOiB7fSwgImVuY3J5cHRpb25fa2V5X2lkIjogbnVsbCwgImNyZWF0ZWRfYXQiOiAiMjAyMS0xMi0xMVQyMTowNToxNVoiLCAidXBkYXRlZF9hdCI6ICIyMDIxLTEyLTExVDIxOjA1OjE5WiIsICJkZWxldGVkX2F0IjogbnVsbCwgImRlbGV0ZWQiOiBmYWxzZX0=.
Dec 11 21:05:20.561750 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-bc3219f6-a390-4e1a-9a68-b11f203904fa tempest-VolumesBackupsAdminTest-1846777574 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:20.684020 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-bc3219f6-a390-4e1a-9a68-b11f203904fa tempest-VolumesBackupsAdminTest-1846777574 None] Import record id 507b7c3c-1102-4e8f-8c25-30f9d3da238c metadata from driver finished.
Dec 11 21:05:23.339627 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Restore backup started, backup: ad504e7d-6609-46da-9e6d-d50578f9abe2 volume: b7db60e7-b0d8-41c9-9119-7101f0a2232d.
Dec 11 21:05:23.390501 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:23.390501 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.412066 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.023s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:23.413144 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[39a0589d-6ffc-42e0-81e4-c7432eae4de1]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:23.416126 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.433576 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[bcf67d61-e7b0-4b5f-a6de-86a3f9a6ab34]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:05:23.435822 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:05:23.437939 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcf67d61-e7b0-4b5f-a6de-86a3f9a6ab34]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:23.440083 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.469704 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e287ffdf-198c-4b8c-a9fb-e3dc98e0ffc6]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:05:23.470316 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:05:23.472509 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e287ffdf-198c-4b8c-a9fb-e3dc98e0ffc6]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:23.476461 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.509282 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.035s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:23.510863 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[729ba6a7-7d06-44ff-9c77-8a5c04873250]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:23.515233 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.530046 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.016s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:23.530410 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[193ad25a-159f-4aee-97d2-5375ed2fd0ee]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:23.532835 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.543814 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.011s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:23.544270 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab25be94-093f-4561-8d26-fda50d3c8a2f]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:23.545956 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:23.572214 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] CMD "nvme version" returned: 0 in 0.026s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:23.575502 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] <== get_connector_properties: return (187ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:24.465426 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:05:24.467429 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2663be20>, 'connection_properties': {'name': 'volumes/volume-b7db60e7-b0d8-41c9-9119-7101f0a2232d', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b7db60e7-b0d8-41c9-9119-7101f0a2232d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:24.468716 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:05:24.572602 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] <== connect_volume: return (104ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c266043a0>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:24.584115 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:24.601129 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Starting restore from Ceph backup=ad504e7d-6609-46da-9e6d-d50578f9abe2 to volume=volume-b7db60e7-b0d8-41c9-9119-7101f0a2232d {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Dec 11 21:05:24.759894 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Looking for snapshot of backup base 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:05:24.761007 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Found snapshot 'backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:05:24.773833 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Volume file is RBD. {{(pid=134086) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Dec 11 21:05:24.831899 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Attempting differential restore. {{(pid=134086) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Dec 11 21:05:24.832322 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Attempting incremental restore from base='volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' snap='backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625' {{(pid=134086) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Dec 11 21:05:24.832993 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Performing differential transfer from 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' to 'volume-b7db60e7-b0d8-41c9-9119-7101f0a2232d' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:05:24.833936 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2@backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:05:24.834574 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_4vxr5vvh --pool volumes - volumes/volume-b7db60e7-b0d8-41c9-9119-7101f0a2232d' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:05:26.076790 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Restore transfer completed in 1.2433s {{(pid=134086) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Dec 11 21:05:26.080352 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Dec 11 21:05:26.139923 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Restoring volume base metadata {{(pid=134086) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Dec 11 21:05:26.177320 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] No metadata of type 'volume-metadata' to restore {{(pid=134086) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Dec 11 21:05:26.177654 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=134086) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Dec 11 21:05:26.188020 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Restore to volume b7db60e7-b0d8-41c9-9119-7101f0a2232d finished successfully. {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Dec 11 21:05:26.190408 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2663be20>, 'connection_properties': {'name': 'volumes/volume-b7db60e7-b0d8-41c9-9119-7101f0a2232d', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'b7db60e7-b0d8-41c9-9119-7101f0a2232d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c266043a0>}, 'force': True, 'ignore_errors': False}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:26.210795 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] <== disconnect_volume: return (19ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:26.277637 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.manager [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Encryption key ID for volume b7db60e7-b0d8-41c9-9119-7101f0a2232d already matches encryption key ID in backup ad504e7d-6609-46da-9e6d-d50578f9abe2. {{(pid=134086) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Dec 11 21:05:26.327987 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-15f6d006-8fa7-412a-a50c-40ad11c3414c tempest-VolumesBackupsAdminTest-953270115 None] Finished restoring backup ad504e7d-6609-46da-9e6d-d50578f9abe2 to volume b7db60e7-b0d8-41c9-9119-7101f0a2232d.
Dec 11 21:05:26.911098 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Delete backup started, backup: 507b7c3c-1102-4e8f-8c25-30f9d3da238c.
Dec 11 21:05:26.916461 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Delete started for backup=507b7c3c-1102-4e8f-8c25-30f9d3da238c {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:05:26.918549 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Trying diff format basename='volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' for backup base image of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:05:27.021535 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Looking for snapshot of backup base 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:05:27.022044 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Backup '507b7c3c-1102-4e8f-8c25-30f9d3da238c' has no snapshot {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}}
Dec 11 21:05:27.022446 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] No backup snapshot to delete {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Dec 11 21:05:27.053817 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Backup base image of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e still has 1 snapshots so skipping base image delete.
Dec 11 21:05:27.157040 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Metadata backup object 'backup.507b7c3c-1102-4e8f-8c25-30f9d3da238c.meta' not found - ignoring {{(pid=134086) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}}
Dec 11 21:05:27.161257 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Delete of backup '507b7c3c-1102-4e8f-8c25-30f9d3da238c' for volume 'eadf804a-f17e-4ea4-b992-c925e6e57a8e' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:05:27.168144 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:27.273041 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Created reservations ['ffb0d01e-54f7-4856-a65c-7e11c2e93514', 'f145723c-8065-44d3-b044-85b2929f5e80'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:05:27.350835 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-5ae64902-0746-4cca-af84-d946564c5b85 tempest-VolumesBackupsAdminTest-1846777574 None] Delete backup finished, backup 507b7c3c-1102-4e8f-8c25-30f9d3da238c deleted.
Dec 11 21:05:28.060192 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Delete backup started, backup: ad504e7d-6609-46da-9e6d-d50578f9abe2.
Dec 11 21:05:28.081163 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Delete started for backup=ad504e7d-6609-46da-9e6d-d50578f9abe2 {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:05:28.082081 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Trying diff format basename='volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' for backup base image of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:05:28.179161 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Looking for snapshot of backup base 'volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:05:28.180716 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Found snapshot 'backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:05:28.181943 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Deleting backup snapshot='backup.ad504e7d-6609-46da-9e6d-d50578f9abe2.snap.1639256716.5155625' {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Dec 11 21:05:29.069332 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Deleting backup base image='volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e.
Dec 11 21:05:29.605644 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Base backup image='volume-eadf804a-f17e-4ea4-b992-c925e6e57a8e.backup.ad504e7d-6609-46da-9e6d-d50578f9abe2' of volume eadf804a-f17e-4ea4-b992-c925e6e57a8e deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:05:29.691563 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Delete of backup 'ad504e7d-6609-46da-9e6d-d50578f9abe2' for volume 'eadf804a-f17e-4ea4-b992-c925e6e57a8e' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:05:29.700333 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:29.743461 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Created reservations ['6d71d450-f843-4a3d-bcc6-2630910010d4', 'bc170291-5371-4131-98dc-bb1eeb9d2294'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:05:29.783312 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-e140087f-0862-4a55-b1f4-b9d957e20233 tempest-VolumesBackupsAdminTest-953270115 None] Delete backup finished, backup ad504e7d-6609-46da-9e6d-d50578f9abe2 deleted.
Dec 11 21:05:32.062872 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Create backup started, backup: 21b5d108-c255-4f05-b5c2-0637b5ea5cb8 volume: a3673216-b0df-4687-99ed-25ae706253c1.
Dec 11 21:05:32.073539 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:32.078243 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.096297 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.020s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:32.096599 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[972ad58b-3566-40d1-9044-d6e556f5159f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:32.102661 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.109451 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[af4df5be-6c2a-4e4b-adb4-4c2b03bcd56f]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:05:32.110100 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:05:32.111765 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[af4df5be-6c2a-4e4b-adb4-4c2b03bcd56f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:32.112503 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.120967 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[cc705df1-a5ea-49ea-8dee-e784ce81241b]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:05:32.121784 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:05:32.123510 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc705df1-a5ea-49ea-8dee-e784ce81241b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:32.125121 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.140865 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.015s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:32.141227 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8db4565e-7d6c-42cd-9b03-86dca2172d17]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:32.142471 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.152844 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:32.153631 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e697e82-d3a6-49f1-bd8c-84e8036e8053]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:32.155403 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.170595 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.015s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:32.171047 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f498bd04-8ba1-4bd2-b137-cf10fde65115]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:05:32.171956 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:05:32.192774 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] CMD "nvme version" returned: 0 in 0.021s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:05:32.194010 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:33.406570 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:05:33.408925 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26548850>, 'connection_properties': {'name': 'volumes/volume-a3673216-b0df-4687-99ed-25ae706253c1', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a3673216-b0df-4687-99ed-25ae706253c1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:33.410460 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:05:33.507872 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] <== connect_volume: return (97ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2651c790>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:33.523081 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:33.565252 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Starting backup of volume='a3673216-b0df-4687-99ed-25ae706253c1'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:05:33.567716 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Volume file is RBD: attempting optimized backup {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Dec 11 21:05:33.623661 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Creating base image 'volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' {{(pid=134086) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Dec 11 21:05:33.624320 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:05:33.727228 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Using --from-snap 'None' for incremental backup of volume a3673216-b0df-4687-99ed-25ae706253c1. {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Dec 11 21:05:33.728244 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Creating backup snapshot='backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8.snap.1639256733.7275844' {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Dec 11 21:05:34.006938 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Performing differential transfer from 'volume-a3673216-b0df-4687-99ed-25ae706253c1' to 'volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:05:34.007760 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_q_2yts11 --pool volumes volumes/volume-a3673216-b0df-4687-99ed-25ae706253c1@backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8.snap.1639256733.7275844 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:05:34.008667 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:05:35.087837 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Differential backup transfer completed in 1.0793s {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Dec 11 21:05:35.087837 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:05:35.120133 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:05:35.120809 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:05:35.129322 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:05:35.129822 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:05:35.153625 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] No metadata type 'volume-glance-metadata' available {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Dec 11 21:05:35.154226 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Backing up metadata for volume a3673216-b0df-4687-99ed-25ae706253c1. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:05:35.229598 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Backup '21b5d108-c255-4f05-b5c2-0637b5ea5cb8' of volume a3673216-b0df-4687-99ed-25ae706253c1 finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:05:35.231706 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26548850>, 'connection_properties': {'name': 'volumes/volume-a3673216-b0df-4687-99ed-25ae706253c1', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'a3673216-b0df-4687-99ed-25ae706253c1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2651c790>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:05:35.293308 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] <== disconnect_volume: return (60ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:05:35.371670 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-cd2acf9e-c3ca-4e36-bb97-00b244ac2a05 tempest-VolumesBackupsAdminTest-953270115 None] Create backup finished. backup: 21b5d108-c255-4f05-b5c2-0637b5ea5cb8.
Dec 11 21:05:36.374279 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-f3d4ddc4-2dfe-4896-9ed9-eb90d6339775 tempest-VolumesBackupsAdminTest-1846777574 None] Reset backup status started, backup_id: 21b5d108-c255-4f05-b5c2-0637b5ea5cb8, status: error.
Dec 11 21:05:36.375090 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-f3d4ddc4-2dfe-4896-9ed9-eb90d6339775 tempest-VolumesBackupsAdminTest-1846777574 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver.
Dec 11 21:05:37.510691 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Delete backup started, backup: 21b5d108-c255-4f05-b5c2-0637b5ea5cb8.
Dec 11 21:05:37.513391 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Delete started for backup=21b5d108-c255-4f05-b5c2-0637b5ea5cb8 {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:05:37.514022 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Trying diff format basename='volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' for backup base image of volume a3673216-b0df-4687-99ed-25ae706253c1. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:05:37.599981 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Looking for snapshot of backup base 'volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:05:37.600761 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Found snapshot 'backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8.snap.1639256733.7275844' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:05:37.601483 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Deleting backup snapshot='backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8.snap.1639256733.7275844' {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Dec 11 21:05:37.991338 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-2c9f83c9-e9b4-413e-978e-77caa8037e44 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:05:38.111630 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Deleting backup base image='volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' of volume a3673216-b0df-4687-99ed-25ae706253c1.
Dec 11 21:05:38.604967 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Base backup image='volume-a3673216-b0df-4687-99ed-25ae706253c1.backup.21b5d108-c255-4f05-b5c2-0637b5ea5cb8' of volume a3673216-b0df-4687-99ed-25ae706253c1 deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:05:38.688405 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Delete of backup '21b5d108-c255-4f05-b5c2-0637b5ea5cb8' for volume 'a3673216-b0df-4687-99ed-25ae706253c1' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:05:38.728935 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Created reservations ['5af233b7-888e-4ae7-ba32-1061b45a6a9d', '71661d95-8f9d-4d5a-835a-7ff97ed8a577'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:05:38.803693 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-c293ef31-3c72-470b-8b97-7b11da6c473f tempest-VolumesBackupsAdminTest-953270115 None] Delete backup finished, backup 21b5d108-c255-4f05-b5c2-0637b5ea5cb8 deleted.
Dec 11 21:05:45.842563 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:05:45.843269 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:06:05.614932 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Delete backup started, backup: 750f3b0c-4514-4200-82f6-5f96dd472cdc.
Dec 11 21:06:05.617246 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Delete started for backup=750f3b0c-4514-4200-82f6-5f96dd472cdc {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:06:05.617793 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Trying diff format basename='volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' for backup base image of volume 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:06:05.725292 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Looking for snapshot of backup base 'volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:06:05.728287 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Found snapshot 'backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:06:05.729256 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Deleting backup snapshot='backup.750f3b0c-4514-4200-82f6-5f96dd472cdc.snap.1639256643.7980025' {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Dec 11 21:06:06.158159 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Deleting backup base image='volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' of volume 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.
Dec 11 21:06:07.418775 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Base backup image='volume-7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d.backup.750f3b0c-4514-4200-82f6-5f96dd472cdc' of volume 7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:06:07.530551 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Delete of backup '750f3b0c-4514-4200-82f6-5f96dd472cdc' for volume '7b7fe2cc-e7c8-4e38-a28e-c31004f8fe0d' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:06:07.536755 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:06:07.594726 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Created reservations ['9631f154-22a1-41d7-a3c2-feadacba27bd', '04cad384-11a5-4d5f-a05a-6e7be417b093'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:06:07.642622 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-b2c63916-b767-4099-bc97-0983596b97e9 tempest-TestVolumeBackupRestore-1162231384 None] Delete backup finished, backup 750f3b0c-4514-4200-82f6-5f96dd472cdc deleted.
Dec 11 21:06:16.767638 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Create backup started, backup: 95feb87e-ef57-46dc-8525-244568fa8eed volume: 78342019-478e-43a5-8d0b-38bf722c00ec.
Dec 11 21:06:16.776340 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:06:16.777673 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.786595 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:06:16.786895 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9948649e-807c-4c69-9873-915f2214f147]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:06:16.788761 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.795129 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e3cb8e68-ce3b-4478-870a-4eb400733bd5]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:06:16.795766 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:06:16.797175 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3cb8e68-ce3b-4478-870a-4eb400733bd5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:06:16.798395 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.806060 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a0eeb592-3778-48f8-94f2-71ea555487bd]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:06:16.814450 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0eeb592-3778-48f8-94f2-71ea555487bd]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:06:16.816636 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.825708 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:06:16.826056 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cea41e13-c0f8-41c1-9fe8-50f89470be9a]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:06:16.827203 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.836147 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:06:16.836556 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7eccde28-d430-4853-bf69-b2e325792075]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:06:16.837660 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.845228 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:06:16.845562 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9d1f657-c952-4231-83ae-74c208099de7]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:06:16.846368 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:06:16.863930 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] CMD "nvme version" returned: 0 in 0.017s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:06:16.865015 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] <== get_connector_properties: return (87ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:06:17.997150 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-2c9f83c9-e9b4-413e-978e-77caa8037e44 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:06:18.001996 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:06:18.003829 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26525c70>, 'connection_properties': {'name': 'volumes/volume-78342019-478e-43a5-8d0b-38bf722c00ec', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '78342019-478e-43a5-8d0b-38bf722c00ec', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:06:18.005207 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:06:18.093619 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] <== connect_volume: return (88ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2652da60>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:06:18.140094 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Starting backup of volume='78342019-478e-43a5-8d0b-38bf722c00ec'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:06:18.142208 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Volume file is RBD: attempting optimized backup {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Dec 11 21:06:18.179276 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Creating base image 'volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' {{(pid=134086) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Dec 11 21:06:18.179689 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:06:18.239619 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Using --from-snap 'None' for incremental backup of volume 78342019-478e-43a5-8d0b-38bf722c00ec. {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Dec 11 21:06:18.240380 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Creating backup snapshot='backup.95feb87e-ef57-46dc-8525-244568fa8eed.snap.1639256778.2399209' {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Dec 11 21:06:19.267750 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Performing differential transfer from 'volume-78342019-478e-43a5-8d0b-38bf722c00ec' to 'volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:06:19.268544 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_2kn6l5j_ --pool volumes volumes/volume-78342019-478e-43a5-8d0b-38bf722c00ec@backup.95feb87e-ef57-46dc-8525-244568fa8eed.snap.1639256778.2399209 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:06:19.268544 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:06:20.352806 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Differential backup transfer completed in 1.0847s {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Dec 11 21:06:20.354067 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:06:20.377928 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:06:20.379126 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:06:20.395422 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:06:20.396814 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:06:20.411832 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] No metadata type 'volume-glance-metadata' available {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Dec 11 21:06:20.416097 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Backing up metadata for volume 78342019-478e-43a5-8d0b-38bf722c00ec. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:06:20.464365 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Backup '95feb87e-ef57-46dc-8525-244568fa8eed' of volume 78342019-478e-43a5-8d0b-38bf722c00ec finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:06:20.466300 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26525c70>, 'connection_properties': {'name': 'volumes/volume-78342019-478e-43a5-8d0b-38bf722c00ec', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '78342019-478e-43a5-8d0b-38bf722c00ec', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2652da60>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:06:20.509167 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] <== disconnect_volume: return (41ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:06:20.585456 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-c2327310-c652-4ed1-a0ad-776a7d38ce4a tempest-VolumesBackupsV39Test-978310112 None] Create backup finished. backup: 95feb87e-ef57-46dc-8525-244568fa8eed.
Dec 11 21:06:21.172802 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Delete backup started, backup: 95feb87e-ef57-46dc-8525-244568fa8eed.
Dec 11 21:06:21.175409 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Delete started for backup=95feb87e-ef57-46dc-8525-244568fa8eed {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:06:21.176104 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Trying diff format basename='volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' for backup base image of volume 78342019-478e-43a5-8d0b-38bf722c00ec. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:06:21.328743 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Looking for snapshot of backup base 'volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:06:21.329256 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Found snapshot 'backup.95feb87e-ef57-46dc-8525-244568fa8eed.snap.1639256778.2399209' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:06:21.329758 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Deleting backup snapshot='backup.95feb87e-ef57-46dc-8525-244568fa8eed.snap.1639256778.2399209' {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Dec 11 21:06:22.105853 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Deleting backup base image='volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' of volume 78342019-478e-43a5-8d0b-38bf722c00ec.
Dec 11 21:06:22.512883 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Base backup image='volume-78342019-478e-43a5-8d0b-38bf722c00ec.backup.95feb87e-ef57-46dc-8525-244568fa8eed' of volume 78342019-478e-43a5-8d0b-38bf722c00ec deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:06:22.605809 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Delete of backup '95feb87e-ef57-46dc-8525-244568fa8eed' for volume '78342019-478e-43a5-8d0b-38bf722c00ec' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:06:22.618414 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:06:22.656047 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Created reservations ['1294f48e-3019-458c-a979-d0d3fc741100', '06189e4a-da72-4448-95d3-21c00b74c9cf'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:06:22.726709 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-fcbb4209-fccf-418b-a886-50041bd2c44c tempest-VolumesBackupsV39Test-978310112 None] Delete backup finished, backup 95feb87e-ef57-46dc-8525-244568fa8eed deleted.
Dec 11 21:06:45.839550 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:06:45.840106 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:07:45.843277 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:07:45.844374 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:08:45.845827 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:08:45.847342 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:09:45.842985 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:09:45.843697 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:10:45.841055 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:10:45.841748 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:11:05.391630 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Create backup started, backup: 1fa0cd39-35e6-4e2e-8601-5220ba11e46f volume: a5ad3712-ba49-4336-9ea6-4de8b933fc6f snapshot: 0e151072-24bc-4b61-996a-553a6eb49ce8.
Dec 11 21:11:05.401101 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:11:05.406902 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.416182 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:11:05.416545 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8b9a611-35fb-4e32-b2a4-ecc9eb4d6c7d]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:11:05.423317 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.435310 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d5160fb7-a254-410a-b708-bbfc9627b243]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:11:05.435890 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:11:05.437244 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5160fb7-a254-410a-b708-bbfc9627b243]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:11:05.438330 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.447793 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[73e5dbc4-d71e-40c4-b7ae-cfd867ac61ef]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:11:05.448229 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:11:05.449614 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[73e5dbc4-d71e-40c4-b7ae-cfd867ac61ef]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:11:05.450810 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.463621 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.013s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:11:05.465591 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[862c13d3-851d-4d26-ad60-db46df60b230]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:11:05.468579 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.478044 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.011s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:11:05.479145 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[98732e21-da27-4dea-8177-c51ff5d7d762]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:11:05.481243 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.489800 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:11:05.490326 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3383375d-2af0-407e-acd0-6008be4b017a]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:11:05.491261 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:11:05.513172 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] CMD "nvme version" returned: 0 in 0.022s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:11:05.514616 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] <== get_connector_properties: return (112ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:11:06.821847 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:11:06.823309 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2685edf0>, 'connection_properties': {'name': 'volumes/volume-155f9e95-a05d-4320-bbcf-50014d909d4c', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '155f9e95-a05d-4320-bbcf-50014d909d4c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:11:06.824984 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:11:06.907861 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] <== connect_volume: return (83ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2653aaf0>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:11:06.918969 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:11:06.944475 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Starting backup of volume='a5ad3712-ba49-4336-9ea6-4de8b933fc6f'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:11:06.978379 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:11:06.978757 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Creating backup base image='volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.base' for volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f. {{(pid=134086) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}}
Dec 11 21:11:07.017348 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Copying data from volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f. {{(pid=134086) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}}
Dec 11 21:11:07.035929 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferring data between 'volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f' and 'volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.base' {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}}
Dec 11 21:11:07.036541 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] 8 chunks of 134217728 bytes to be transferred {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}}
Dec 11 21:11:08.419150 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 1 of 8 (94851K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:09.557776 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 2 of 8 (115267K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:10.595527 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 3 of 8 (126552K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:11.629558 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 4 of 8 (126830K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:12.834574 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 5 of 8 (108827K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:13.980322 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 6 of 8 (114589K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:15.165405 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 7 of 8 (110657K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:16.275421 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Transferred chunk 8 of 8 (118147K/s) {{(pid=134086) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Dec 11 21:11:16.320260 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:11:16.332269 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:11:16.332754 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:11:16.340625 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:11:16.340625 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:11:16.346260 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] No metadata type 'volume-glance-metadata' available {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Dec 11 21:11:16.347189 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Backing up metadata for volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:11:16.389023 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Backup '1fa0cd39-35e6-4e2e-8601-5220ba11e46f' of volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:11:16.391199 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c2685edf0>, 'connection_properties': {'name': 'volumes/volume-155f9e95-a05d-4320-bbcf-50014d909d4c', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '155f9e95-a05d-4320-bbcf-50014d909d4c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2653aaf0>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:11:16.407919 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] <== disconnect_volume: return (15ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:11:16.501381 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-f5ecd2a5-c97b-4af3-bc3c-8999ff9c4d2c tempest-VolumesSnapshotTestJSON-970988873 None] Create backup finished. backup: 1fa0cd39-35e6-4e2e-8601-5220ba11e46f.
Dec 11 21:11:16.836422 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Delete backup started, backup: 1fa0cd39-35e6-4e2e-8601-5220ba11e46f.
Dec 11 21:11:16.838214 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Delete started for backup=1fa0cd39-35e6-4e2e-8601-5220ba11e46f {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:11:16.838735 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Trying diff format basename='volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.1fa0cd39-35e6-4e2e-8601-5220ba11e46f' for backup base image of volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:11:16.871608 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Image 'volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.1fa0cd39-35e6-4e2e-8601-5220ba11e46f' not found - trying diff format name {{(pid=134086) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}}
Dec 11 21:11:16.894273 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Looking for snapshot of backup base 'volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.base' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:11:16.894686 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Backup base 'volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.base' has no snapshots {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}}
Dec 11 21:11:16.895060 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] No backup snapshot to delete {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Dec 11 21:11:16.901716 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Deleting backup base image='volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.base' of volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f.
Dec 11 21:11:17.605540 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Base backup image='volume-a5ad3712-ba49-4336-9ea6-4de8b933fc6f.backup.base' of volume a5ad3712-ba49-4336-9ea6-4de8b933fc6f deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:11:17.654100 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Delete of backup '1fa0cd39-35e6-4e2e-8601-5220ba11e46f' for volume 'a5ad3712-ba49-4336-9ea6-4de8b933fc6f' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:11:17.660081 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:11:17.684330 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Created reservations ['f093edb4-63b9-4a70-a673-b370f9149e13', '01341e97-db98-4533-a422-40a9f40292bc'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:11:17.708497 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-0082b1dd-e86e-48ce-aca5-e796dff94a4a tempest-VolumesSnapshotTestJSON-970988873 None] Delete backup finished, backup 1fa0cd39-35e6-4e2e-8601-5220ba11e46f deleted.
Dec 11 21:11:45.838738 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:11:45.839387 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:12:45.839897 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:12:45.840623 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:13:45.844502 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:13:45.845631 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:14:45.844445 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:14:45.847257 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:15:45.849239 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:15:45.850643 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:16:45.850198 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:16:45.850944 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:17:45.854780 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:17:45.858403 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:18:45.860142 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:18:45.862622 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:19:45.859268 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:19:45.859844 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:20:45.857784 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:20:45.858701 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:21:45.854326 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:21:45.855223 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:22:45.849900 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:22:45.850521 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:23:17.986981 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Create backup started, backup: 3e7c431c-4c42-40e6-9c59-37dcad8a8e64 volume: c91a9a74-a7d0-4006-9f18-589c5f96cdf4.
Dec 11 21:23:17.995342 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:17.996780 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.005717 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:18.006077 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb1e1023-3d17-42af-b7a9-7c7788177201]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:18.008161 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.013924 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e3781577-af83-4d45-a12c-1aa94b425ff6]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:23:18.014528 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:23:18.015863 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3781577-af83-4d45-a12c-1aa94b425ff6]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:18.015949 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.020989 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[732f1c8f-22f6-4f33-9c15-4c13cd8a6aa9]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:23:18.021530 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:23:18.022844 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[732f1c8f-22f6-4f33-9c15-4c13cd8a6aa9]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:18.023628 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.046182 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:18.046421 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[247b360f-a917-49f8-9db8-8f4a3a127102]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:18.047751 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.058977 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.011s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:18.059331 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[11458137-b30e-41b9-a401-d0f39ccd5d6f]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:18.060449 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.067190 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:18.067478 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[596d703b-3576-4e29-b40e-2aca4138c1d7]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:18.068199 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:18.090796 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] CMD "nvme version" returned: 0 in 0.022s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:18.092238 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] <== get_connector_properties: return (96ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:18.148035 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-2c9f83c9-e9b4-413e-978e-77caa8037e44 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:23:18.820546 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:23:18.821962 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26982220>, 'connection_properties': {'name': 'volumes/volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'c91a9a74-a7d0-4006-9f18-589c5f96cdf4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:18.823045 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:23:18.862625 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] <== connect_volume: return (40ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2654b520>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:18.879708 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Starting backup of volume='c91a9a74-a7d0-4006-9f18-589c5f96cdf4'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:23:18.880690 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Volume file is RBD: attempting optimized backup {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Dec 11 21:23:18.901289 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Creating base image 'volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' {{(pid=134086) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Dec 11 21:23:18.901691 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:23:18.928120 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Using --from-snap 'None' for incremental backup of volume c91a9a74-a7d0-4006-9f18-589c5f96cdf4. {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Dec 11 21:23:18.928527 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Creating backup snapshot='backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64.snap.1639257798.9281428' {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Dec 11 21:23:19.370935 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Performing differential transfer from 'volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4' to 'volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:23:19.371590 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_twazjunb --pool volumes volumes/volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4@backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64.snap.1639257798.9281428 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:23:19.371712 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:23:20.402772 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Differential backup transfer completed in 1.0314s {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Dec 11 21:23:20.403427 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:23:20.416675 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:23:20.417084 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:23:20.423171 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:23:20.423551 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:23:20.429485 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] No metadata type 'volume-glance-metadata' available {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Dec 11 21:23:20.430013 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Backing up metadata for volume c91a9a74-a7d0-4006-9f18-589c5f96cdf4. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:23:20.461103 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Backup '3e7c431c-4c42-40e6-9c59-37dcad8a8e64' of volume c91a9a74-a7d0-4006-9f18-589c5f96cdf4 finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:23:20.462717 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26982220>, 'connection_properties': {'name': 'volumes/volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'c91a9a74-a7d0-4006-9f18-589c5f96cdf4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2654b520>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:20.471885 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] <== disconnect_volume: return (8ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:20.528599 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-001c9e7c-ed13-4689-9d7f-78c15000dfbb tempest-VolumesBackupsTest-846679880 None] Create backup finished. backup: 3e7c431c-4c42-40e6-9c59-37dcad8a8e64.
Dec 11 21:23:21.125926 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Delete backup started, backup: 3e7c431c-4c42-40e6-9c59-37dcad8a8e64.
Dec 11 21:23:21.127505 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Delete started for backup=3e7c431c-4c42-40e6-9c59-37dcad8a8e64 {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:23:21.127922 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Trying diff format basename='volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' for backup base image of volume c91a9a74-a7d0-4006-9f18-589c5f96cdf4. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:23:21.176253 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Looking for snapshot of backup base 'volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:23:21.176641 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Found snapshot 'backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64.snap.1639257798.9281428' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:23:21.177038 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Deleting backup snapshot='backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64.snap.1639257798.9281428' {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Dec 11 21:23:21.422133 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Deleting backup base image='volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' of volume c91a9a74-a7d0-4006-9f18-589c5f96cdf4.
Dec 11 21:23:21.581880 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Base backup image='volume-c91a9a74-a7d0-4006-9f18-589c5f96cdf4.backup.3e7c431c-4c42-40e6-9c59-37dcad8a8e64' of volume c91a9a74-a7d0-4006-9f18-589c5f96cdf4 deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:23:21.615703 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Delete of backup '3e7c431c-4c42-40e6-9c59-37dcad8a8e64' for volume 'c91a9a74-a7d0-4006-9f18-589c5f96cdf4' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:23:21.621956 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:23:21.643756 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Created reservations ['87b5cb23-9509-4654-8a80-a310a3afb5aa', '3f2b7f56-7e4d-42cd-b42b-0d1938b8718c'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:23:21.668723 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-8118a008-3b54-4c57-83fa-b65d555258d2 tempest-VolumesBackupsTest-846679880 None] Delete backup finished, backup 3e7c431c-4c42-40e6-9c59-37dcad8a8e64 deleted.
Dec 11 21:23:30.804734 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Create backup started, backup: 5d4e07e5-41a5-4ed5-b597-1a8352d12130 volume: 8584d382-fe1d-46c6-b7b7-1345dee565bb.
Dec 11 21:23:30.812332 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:30.813694 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.821184 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:30.821528 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a041c72a-5358-4fcc-bfde-094a7ba71677]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:30.822702 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.828010 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[63d9fd96-8ffa-4fe0-93cc-d20562eaffac]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:23:30.828580 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:23:30.829771 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[63d9fd96-8ffa-4fe0-93cc-d20562eaffac]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:30.830111 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.835750 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[07bfa79b-2937-4fce-874f-72150e1c8c0a]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:23:30.836361 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:23:30.837529 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[07bfa79b-2937-4fce-874f-72150e1c8c0a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:30.838124 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.846169 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:30.846580 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[88fdaa5c-e20a-4982-bf37-51db7934119d]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:30.847641 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.855937 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:30.856277 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7f2c617-3109-46b7-a7ce-fc5591061935]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:30.857323 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.863614 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:30.863940 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5c70c38-658e-44ca-87bd-4604c18511ca]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:30.864692 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:30.883725 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:30.884832 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] <== get_connector_properties: return (71ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:31.574098 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:23:31.575589 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26525ac0>, 'connection_properties': {'name': 'volumes/volume-8584d382-fe1d-46c6-b7b7-1345dee565bb', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8584d382-fe1d-46c6-b7b7-1345dee565bb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:31.576689 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:23:31.641234 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] <== connect_volume: return (65ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c25d27430>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:31.649318 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:23:31.666468 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Starting backup of volume='8584d382-fe1d-46c6-b7b7-1345dee565bb'. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Dec 11 21:23:31.667385 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Volume file is RBD: attempting optimized backup {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Dec 11 21:23:31.686767 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Creating base image 'volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' {{(pid=134086) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Dec 11 21:23:31.687173 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134086) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Dec 11 21:23:31.716693 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Using --from-snap 'None' for incremental backup of volume 8584d382-fe1d-46c6-b7b7-1345dee565bb. {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Dec 11 21:23:31.717103 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Creating backup snapshot='backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192' {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Dec 11 21:23:31.802176 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Performing differential transfer from 'volume-8584d382-fe1d-46c6-b7b7-1345dee565bb' to 'volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:23:31.802635 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_z3iuxb2f --pool volumes volumes/volume-8584d382-fe1d-46c6-b7b7-1345dee565bb@backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:23:31.803021 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:23:36.060284 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Differential backup transfer completed in 4.2577s {{(pid=134086) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Dec 11 21:23:36.060944 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Getting metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Dec 11 21:23:36.074185 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Dec 11 21:23:36.074631 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Getting metadata type 'volume-metadata' {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Dec 11 21:23:36.080983 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] No metadata type 'volume-metadata' available {{(pid=134086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Dec 11 21:23:36.081373 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Getting metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Dec 11 21:23:36.088753 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Dec 11 21:23:36.089322 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Backing up metadata for volume 8584d382-fe1d-46c6-b7b7-1345dee565bb. {{(pid=134086) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Dec 11 21:23:36.123885 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Backup '5d4e07e5-41a5-4ed5-b597-1a8352d12130' of volume 8584d382-fe1d-46c6-b7b7-1345dee565bb finished. {{(pid=134086) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Dec 11 21:23:36.125539 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26525ac0>, 'connection_properties': {'name': 'volumes/volume-8584d382-fe1d-46c6-b7b7-1345dee565bb', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '8584d382-fe1d-46c6-b7b7-1345dee565bb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c25d27430>}, 'force': True, 'ignore_errors': True}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:36.142892 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] <== disconnect_volume: return (16ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:36.205763 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-635b8799-8414-41cf-a116-ec646e41d400 tempest-VolumesBackupsTest-846679880 None] Create backup finished. backup: 5d4e07e5-41a5-4ed5-b597-1a8352d12130.
Dec 11 21:23:38.275379 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Restore backup started, backup: 5d4e07e5-41a5-4ed5-b597-1a8352d12130 volume: 99280ea9-5635-4ba2-a906-d30233819d72.
Dec 11 21:23:38.293953 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.18', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:38.297904 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.304191 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:38.304776 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[100e9163-52fe-463b-85de-d0621e89065d]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:97f02b2dda55\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:38.306603 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.313863 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a584aaf1-a102-460a-8bc2-ed8af185303e]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:23:38.314166 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:23:38.315480 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a584aaf1-a102-460a-8bc2-ed8af185303e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:38.316735 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.327331 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[24b5507d-a5c0-4fba-927a-62e06ba5d044]: [Errno 2] No such file or directory: 'systool' {{(pid=158115) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]: Traceback (most recent call last):
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     ret = func(*f_args, **f_kwargs)
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     return func(*args, **kwargs)
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     return putils.execute(on_execute=on_execute,
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     obj = subprocess.Popen(cmd,
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     self._execute_child(args, executable, preexec_fn, close_fds,
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]:     raise child_exception_type(errno_num, err_msg, err_filename)
Dec 11 21:23:38.327762 ncdv-master-1 cinder-backup[134086]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Dec 11 21:23:38.328998 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[24b5507d-a5c0-4fba-927a-62e06ba5d044]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:38.329950 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.339222 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:38.339783 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[756ebc90-8597-474b-baaa-bd75e580882a]: (4, ('/dev/sda1\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:38.340990 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.351084 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:38.351538 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3aa79bc-d5c9-48e9-b270-e57ed6739d50]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:38.353343 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.361805 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=158115) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:38.362131 ncdv-master-1 cinder-backup[134086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5ad9f28-2c3d-49fd-811f-d5388e36aa3e]: (4, ('2de4ff5c-db05-6442-be67-84c6a6a6e6ad\n', '')) {{(pid=158115) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Dec 11 21:23:38.363018 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Running cmd (subprocess): nvme version {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Dec 11 21:23:38.382938 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_concurrency.processutils [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=134086) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Dec 11 21:23:38.384042 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.utils [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] <== get_connector_properties: return (89ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.18', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:97f02b2dda55', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '2de4ff5c-db05-6442-be67-84c6a6a6e6ad', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:dccb4f2f-119c-49ae-9de6-ee567a89373f'} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:39.071305 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connector [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Factory for rbd on None {{(pid=134086) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Dec 11 21:23:39.072817 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26784400>, 'connection_properties': {'name': 'volumes/volume-99280ea9-5635-4ba2-a906-d30233819d72', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '99280ea9-5635-4ba2-a906-d30233819d72', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:39.073867 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.linuxrbd [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] opening connection to ceph cluster (timeout=-1). {{(pid=134086) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Dec 11 21:23:39.112806 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] <== connect_volume: return (39ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2649d970>} {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:39.119834 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:23:39.130436 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Starting restore from Ceph backup=5d4e07e5-41a5-4ed5-b597-1a8352d12130 to volume=volume-99280ea9-5635-4ba2-a906-d30233819d72 {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Dec 11 21:23:39.203113 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Looking for snapshot of backup base 'volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:23:39.203611 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Found snapshot 'backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:23:39.208998 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Volume file is RBD. {{(pid=134086) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Dec 11 21:23:39.272229 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Attempting differential restore. {{(pid=134086) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Dec 11 21:23:39.272972 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Attempting incremental restore from base='volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' snap='backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192' {{(pid=134086) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Dec 11 21:23:39.273398 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Performing differential transfer from 'volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' to 'volume-99280ea9-5635-4ba2-a906-d30233819d72' {{(pid=134086) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Dec 11 21:23:39.274561 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130@backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192 -' into... {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Dec 11 21:23:39.275108 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_ybrhyk1k --pool volumes - volumes/volume-99280ea9-5635-4ba2-a906-d30233819d72' {{(pid=134086) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Dec 11 21:23:43.230787 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Restore transfer completed in 3.9568s {{(pid=134086) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Dec 11 21:23:43.231831 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Dec 11 21:23:43.253648 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Restoring volume base metadata {{(pid=134086) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Dec 11 21:23:43.273007 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] No metadata of type 'volume-metadata' to restore {{(pid=134086) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Dec 11 21:23:43.273431 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.driver [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Restoring volume glance metadata {{(pid=134086) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Dec 11 21:23:43.408674 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Restore to volume 99280ea9-5635-4ba2-a906-d30233819d72 finished successfully. {{(pid=134086) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Dec 11 21:23:43.410288 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f7c26784400>, 'connection_properties': {'name': 'volumes/volume-99280ea9-5635-4ba2-a906-d30233819d72', 'hosts': ['10.222.0.18'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '99280ea9-5635-4ba2-a906-d30233819d72', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f7c2649d970>}, 'force': True, 'ignore_errors': False}" {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Dec 11 21:23:43.421122 ncdv-master-1 cinder-backup[134086]: DEBUG os_brick.initiator.connectors.rbd [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] <== disconnect_volume: return (9ms) None {{(pid=134086) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Dec 11 21:23:43.479035 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.manager [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Encryption key ID for volume 99280ea9-5635-4ba2-a906-d30233819d72 already matches encryption key ID in backup 5d4e07e5-41a5-4ed5-b597-1a8352d12130. {{(pid=134086) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Dec 11 21:23:43.511749 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-4308be78-9a7b-4c96-922c-17529ceb905f tempest-VolumesBackupsTest-846679880 None] Finished restoring backup 5d4e07e5-41a5-4ed5-b597-1a8352d12130 to volume 99280ea9-5635-4ba2-a906-d30233819d72.
Dec 11 21:23:46.822011 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Delete backup started, backup: 5d4e07e5-41a5-4ed5-b597-1a8352d12130.
Dec 11 21:23:46.823017 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Delete started for backup=5d4e07e5-41a5-4ed5-b597-1a8352d12130 {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Dec 11 21:23:46.823583 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Trying diff format basename='volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' for backup base image of volume 8584d382-fe1d-46c6-b7b7-1345dee565bb. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Dec 11 21:23:46.883207 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Looking for snapshot of backup base 'volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Dec 11 21:23:46.883643 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Found snapshot 'backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192' {{(pid=134086) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Dec 11 21:23:46.884025 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Deleting backup snapshot='backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130.snap.1639257811.7167192' {{(pid=134086) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Dec 11 21:23:47.268385 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Deleting backup base image='volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' of volume 8584d382-fe1d-46c6-b7b7-1345dee565bb.
Dec 11 21:23:47.762133 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Base backup image='volume-8584d382-fe1d-46c6-b7b7-1345dee565bb.backup.5d4e07e5-41a5-4ed5-b597-1a8352d12130' of volume 8584d382-fe1d-46c6-b7b7-1345dee565bb deleted. {{(pid=134086) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Dec 11 21:23:47.798470 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.backup.drivers.ceph [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Delete of backup '5d4e07e5-41a5-4ed5-b597-1a8352d12130' for volume '8584d382-fe1d-46c6-b7b7-1345dee565bb' finished. {{(pid=134086) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Dec 11 21:23:47.804286 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_db.sqlalchemy.engines [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134086) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Dec 11 21:23:47.826787 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.quota [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Created reservations ['64d31514-5178-4a2e-b999-7fdc833b9845', 'a31761f2-38c3-4789-912e-3afb1c26c984'] {{(pid=134086) reserve /opt/stack/cinder/cinder/quota.py:755}}
Dec 11 21:23:47.852999 ncdv-master-1 cinder-backup[134086]: INFO cinder.backup.manager [None req-d439d45c-ed1f-4d12-8960-4f5eba146788 tempest-VolumesBackupsTest-846679880 None] Delete backup finished, backup 5d4e07e5-41a5-4ed5-b597-1a8352d12130 deleted.
Dec 11 21:24:45.848034 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:24:45.848564 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:25:45.846296 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:25:45.846984 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:26:45.852136 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:26:45.852872 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:27:45.848123 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:27:45.848806 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:28:45.853894 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:28:45.854542 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:29:45.859668 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:29:45.860315 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:30:45.855884 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:30:45.856565 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:31:45.861063 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:31:45.861689 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:32:45.865564 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:32:45.866249 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:33:45.870365 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:33:45.871035 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:34:45.874855 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:34:45.875474 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:35:45.879681 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:35:45.880769 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:36:45.885403 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:36:45.886031 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:37:45.884889 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:37:45.885485 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:38:45.880504 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:38:45.881137 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:39:45.885438 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:39:45.886077 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:40:45.882437 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:40:45.882985 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:41:45.887278 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:41:45.887931 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:42:45.892580 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:42:45.893290 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:43:45.898516 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:43:45.899229 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:44:45.894885 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:44:45.895587 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:45:45.900380 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:45:45.901019 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:46:45.905412 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:46:45.906093 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:47:45.910236 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:47:45.910844 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:48:45.915013 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:48:45.915722 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:49:45.919555 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:49:45.920240 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:50:45.924250 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:50:45.924952 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:51:45.920849 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:51:45.921539 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:52:45.925404 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:52:45.926126 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:53:45.931241 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:53:45.931903 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:54:45.929002 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:54:45.929720 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:55:45.925528 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:55:45.926203 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:56:45.930047 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:56:45.930770 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:57:45.934983 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:57:45.935617 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:58:45.940097 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:58:45.940791 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 21:59:45.945982 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 21:59:45.946823 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:00:45.942341 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:00:45.943058 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:01:45.947839 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:01:45.948452 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:02:45.953301 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:02:45.953873 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:03:45.957512 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:03:45.958136 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:04:45.963018 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:04:45.963669 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:05:45.967868 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:05:45.968519 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:06:45.972387 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:06:45.972972 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:07:45.978062 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:07:45.978641 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:08:45.983019 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:08:45.983662 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:09:45.988747 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:09:45.989295 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:10:45.994192 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:10:45.994899 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:11:45.998736 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:11:45.999354 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:12:45.994930 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:12:45.995485 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:13:46.000306 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:13:46.000865 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Dec 11 22:14:46.005437 ncdv-master-1 cinder-backup[134086]: DEBUG oslo_service.periodic_task [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134086) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Dec 11 22:14:46.006047 ncdv-master-1 cinder-backup[134086]: DEBUG cinder.manager [None req-25aaaf37-b589-47dc-932d-f3dc40631c8d None None] Notifying Schedulers of capabilities ... {{(pid=134086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}