-- Logs begin at Sat 2021-11-20 20:01:12 UTC, end at Sat 2021-11-20 22:10:07 UTC. --
Nov 20 20:37:24.214809 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service.
Nov 20 20:37:27.419214 ncdv-master-1 cinder-backup[133857]: INFO cinder.cmd.backup [-] Backup running with 1 processes.
Nov 20 20:37:27.420968 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=133857) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}}
Nov 20 20:37:27.421630 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=133857) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}}
Nov 20 20:37:27.618787 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_db.sqlalchemy.engines [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133857) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 20:37:27.898127 ncdv-master-1 cinder-backup[133857]: INFO cinder.rpc [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version.
Nov 20 20:37:27.900793 ncdv-master-1 cinder-backup[133857]: INFO cinder.rpc [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version.
Nov 20 20:37:27.905392 ncdv-master-1 cinder-backup[133857]: INFO cinder.rpc [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version.
Nov 20 20:37:27.909224 ncdv-master-1 cinder-backup[133857]: INFO cinder.rpc [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version.
Nov 20 20:37:27.913555 ncdv-master-1 cinder-backup[133857]: INFO cinder.rpc [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
Nov 20 20:37:27.918406 ncdv-master-1 cinder-backup[133857]: INFO cinder.rpc [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version.
Nov 20 20:37:27.968101 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_db.sqlalchemy.engines [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133857) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 20:37:27.990636 ncdv-master-1 cinder-backup[133857]: INFO oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Starting 1 workers
Nov 20 20:37:27.996979 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Started child 134303 {{(pid=133857) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Nov 20 20:37:27.998197 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Full set of CONF: {{(pid=133857) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}}
Nov 20 20:37:27.998672 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] ******************************************************************************** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}}
Nov 20 20:37:27.998998 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] Configuration options gathered from: {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}}
Nov 20 20:37:27.999367 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}}
Nov 20 20:37:28.000070 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}}
Nov 20 20:37:28.000424 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] ================================================================================ {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}}
Nov 20 20:37:28.001150 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] allow_availability_zone_fallback = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.001536 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] allow_compression_on_image_upload = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.002101 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] allowed_direct_url_schemes     = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.002553 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] api_rate_limit                 = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.003144 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] auth_strategy                  = keystone {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.003552 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_availability_zone      = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.003943 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_api_class               = cinder.backup.api.API {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.004344 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_chunk_size         = 134217728 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.004987 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_conf               = /etc/ceph/ceph.conf {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.005480 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_image_journals     = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.005903 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_pool               = backups {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.006380 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_stripe_count       = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.006946 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_stripe_unit        = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.007340 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_ceph_user               = cinder-bak {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.007733 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_driver                  = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.008104 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_driver_init_check_interval = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.008483 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_driver_stats_polling_interval = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.008869 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_manager                 = cinder.backup.manager.BackupManager {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.009366 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_max_operations          = 15 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.009760 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_metadata_version        = 2 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.010285 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_name_template           = backup-%s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.010751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_native_threads_pool_size = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.011144 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_object_number_per_notification = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.011487 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_service_inithost_offload = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.011853 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_timer_interval          = 120 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.012238 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_use_same_host           = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.012603 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_use_temp_snapshot       = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.012975 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backup_workers                 = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.013456 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] chap_password                  = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.013829 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] chap_username                  =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.014249 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.014792 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_internal_tenant_project_id = e6c8e0b874174dd4abf198b0af571967 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.015380 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_internal_tenant_user_id = 3c67c5cdca734d5c91c30684265bd9ab {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.015773 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] compression_format             = gzip {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.016195 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] config_dir                     = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.016504 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] config_file                    = ['/etc/cinder/cinder.conf'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.016894 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] config_source                  = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.017249 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] consistencygroup_api_class     = cinder.consistencygroup.api.API {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.017651 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] control_exchange               = openstack {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.018120 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] debug                          = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.018526 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] default_availability_zone      = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.018869 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] default_group_type             = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.019436 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.020555 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] default_volume_type            = ceph {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.020555 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] driver_client_cert             = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.020822 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] driver_client_cert_key         = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.021044 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] driver_data_namespace          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.021301 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] driver_ssl_cert_path           = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.021638 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] driver_ssl_cert_verify         = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.022010 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] driver_use_ssl                 = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.022334 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] enable_new_services            = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.022692 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] enable_unsupported_driver      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.023090 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] enabled_backends               = ['ceph'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.023567 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] enforce_multipath_for_image_xfer = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.023935 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] filter_function                = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.024291 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_api_insecure            = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.024668 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_api_servers             = ['http://10.222.0.41/image'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.025229 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_api_ssl_compression     = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.026277 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_ca_certificates_file    = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.026715 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_catalog_info            = image:glance:publicURL {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.027082 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_certfile                = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.027591 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.027959 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_keyfile                 = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.028320 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_num_retries             = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.028672 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] glance_request_timeout         = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.029023 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] goodness_function              = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.029407 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] graceful_shutdown_timeout      = 5 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.029760 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] group_api_class                = cinder.group.api.API {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.030148 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] host                           = ncdv-master-1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.030751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iet_conf                       = /etc/iet/ietd.conf {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.031129 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_compress_on_upload       = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.031498 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_conversion_address_space_limit = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.031840 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_conversion_cpu_limit     = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.032562 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_conversion_dir           = /opt/stack/data/cinder/conversion {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.032922 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_upload_use_cinder_backend = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.033418 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_upload_use_internal_tenant = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.033796 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_volume_cache_enabled     = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.034154 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_volume_cache_max_count   = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.034512 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] image_volume_cache_max_size_gb = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.034916 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] instance_format                = [instance: %(uuid)s]  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.035693 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.036091 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iscsi_iotype                   = fileio {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.036462 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iscsi_secondary_ip_addresses   = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.036664 ncdv-master-1 cinder-backup[134303]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0)
Nov 20 20:37:28.037155 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iscsi_target_flags             =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.037155 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iscsi_write_cache              = on {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.037531 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iser_helper                    = tgtadm {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.038237 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iser_ip_address                = 10.222.0.41 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.038642 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iser_port                      = 3260 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.039007 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] iser_target_prefix             = iqn.2010-10.org.openstack: {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.039350 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_config_append              = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.039717 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.040060 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_dir                        = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.040399 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_file                       = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.040681 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_options                    = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.041042 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_rotate_interval            = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.042187 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_rotate_interval_type       = days {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.042533 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] log_rotation_type              = none {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.042892 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.043189 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.043695 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.044050 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.044336 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.044696 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] max_age                        = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.045054 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] max_logfile_count              = 30 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.045836 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] max_logfile_size_mb            = 200 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.046263 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] max_over_subscription_ratio    = 20.0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.046659 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] message_reap_interval          = 86400 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.047017 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] message_ttl                    = 2592000 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.047362 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] monkey_patch                   = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.047729 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] monkey_patch_modules           = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.048033 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] my_ip                          = 10.222.0.41 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.048379 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] no_snapshot_gb_quota           = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.048716 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] num_iser_scan_tries            = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.049083 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] num_shell_tries                = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.049441 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] num_volume_device_scan_tries   = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.049782 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] nvmet_ns_id                    = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.050175 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] nvmet_port_id                  = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.050571 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_max_limit                = 1000 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.051025 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_volume_ext_list          = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.051491 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.051862 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_volume_listen            = 0.0.0.0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.052246 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_volume_listen_port       = 8776 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.052695 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_volume_use_ssl           = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.053070 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] osapi_volume_workers           = 2 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.053428 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] per_volume_size_limit          = -1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.053918 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] periodic_fuzzy_delay           = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.054332 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] periodic_interval              = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.054688 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] public_endpoint                = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.055025 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] publish_errors                 = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.055698 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_backup_gigabytes         = 1000 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.056389 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_backups                  = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.056880 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_consistencygroups        = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.057333 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_driver                   = cinder.quota.DbQuotaDriver {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.057696 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_gigabytes                = 1000 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.058090 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_groups                   = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.058433 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_snapshots                = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.058785 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] quota_volumes                  = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.059122 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] rate_limit_burst               = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.059494 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] rate_limit_except_level        = CRITICAL {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.059856 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] rate_limit_interval            = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.060238 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] replication_device             = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.060574 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] report_discard_supported       = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.060935 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] report_interval                = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.061398 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] reservation_clean_interval     = 86400 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.061701 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] reservation_expire             = 86400 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.062039 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] reserved_percentage            = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.062475 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] resource_query_filters_file    = /etc/cinder/resource_filters.json {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.062826 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] restore_discard_excess_bytes   = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.063178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] rootwrap_config                = /etc/cinder/rootwrap.conf {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.063584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] rpc_response_timeout           = 120 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.064069 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] run_external_periodic_tasks    = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.064437 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] scheduler_manager              = cinder.scheduler.manager.SchedulerManager {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.065177 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] scst_target_driver             = iscsi {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.065836 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] scst_target_iqn_name           = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.066310 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_down_time              = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.066751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] snapshot_name_template         = snapshot-%s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.067101 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] split_loggers                  = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.067388 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] state_path                     = /opt/stack/data/cinder {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.067844 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] storage_availability_zone      = nova {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.068203 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] storage_protocol               = iscsi {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.068590 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] syslog_log_facility            = LOG_USER {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.068991 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] target_helper                  = lioadm {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.069452 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] target_ip_address              = 10.222.0.41 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.069864 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] target_port                    = 3260 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.070250 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] target_prefix                  = iqn.2010-10.org.openstack: {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.070639 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] target_protocol                = iscsi {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.071040 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] trace_flags                    = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.071397 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] transfer_api_class             = cinder.transfer.api.API {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.071765 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] transport_url                  = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.072115 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] until_refresh                  = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.072523 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_chap_auth                  = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.072862 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_default_quota_class        = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.073254 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_eventlog                   = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.073611 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_journal                    = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.074296 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_json                       = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.074642 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_multipath_for_image_xfer   = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.074986 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_stderr                     = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.075390 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] use_syslog                     = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.075958 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] verify_glance_signatures       = enabled {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.076473 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_api_class               = cinder.volume.api.API {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.076819 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_backend_name            = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.077229 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_clear                   = zero {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.077584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_clear_ionice            = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.077928 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_clear_size              = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.078302 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_copy_blkio_cgroup_name  = cinder-volume-copy {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.078639 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_copy_bps_limit          = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.078986 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_dd_blocksize            = 1M {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.079403 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_manager                 = cinder.volume.manager.VolumeManager {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.079788 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_name_template           = volume-%s {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.080156 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volume_usage_audit_period      = month {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.080606 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] volumes_dir                    = /opt/stack/data/cinder/volumes {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.080946 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] watch_log_file                 = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Nov 20 20:37:28.081405 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.081809 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.082271 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.enforce_new_defaults = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.082617 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.enforce_scope      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.083001 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.policy_default_rule = default {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.083442 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.083853 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.policy_file        = policy.yaml {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.084448 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.084801 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.085147 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.085487 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.085867 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.086532 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_concurrency.disable_process_locking = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.086919 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_concurrency.lock_path     = /opt/stack/data/cinder {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.087379 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.admin_password = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.087762 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.088699 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.admin_token = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.089042 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.admin_user  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.089426 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_admin_prefix =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.089792 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_host   = 127.0.0.1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.090203 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_port   = 35357 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.090601 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_protocol = https {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.091199 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_section = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.091632 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_type   = password {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.091998 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_uri    = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.092335 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.auth_version = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.092675 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.cache       = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.093047 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.cafile      = /opt/stack/data/ca-bundle.pem {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.093399 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.certfile    = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.093764 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.delay_auth_decision = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.094262 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.094719 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.http_connect_timeout = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.095080 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.095417 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.identity_uri = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.095805 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.include_service_catalog = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.096275 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.insecure    = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.096751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.interface   = public {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.097113 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.keyfile     = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.097472 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.097815 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.098191 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.098536 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.098876 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.099215 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.099594 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.099935 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.100350 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.100687 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.region_name = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.101058 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.101397 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.service_token_roles_required = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.101752 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.service_type = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.102122 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.token_cache_time = 300 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.102498 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.102970 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.backend                  = dogpile.cache.null {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.103338 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.backend_argument         = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.104779 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.config_prefix            = cache.oslo {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.105154 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.debug_cache_backend      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.105498 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.enabled                  = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.105862 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.expiration_time          = 600 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.106350 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_dead_retry      = 300 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.106833 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.107218 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.107595 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_pool_maxsize    = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.108034 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.108502 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.proxies                  = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.tls_allowed_ciphers      = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.tls_cafile               = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.tls_certfile             = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.tls_enabled              = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.116751 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cache.tls_keyfile              = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] coordination.backend_url       = etcd3+http://10.222.0.41:2379 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.backend               = sqlalchemy {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.connection            = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.connection_debug      = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.connection_parameters =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.connection_recycle_time = 3600 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117178 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.connection_trace      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.db_inc_retry_interval = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.db_max_retries        = 20 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.db_max_retry_interval = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.db_retry_interval     = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.max_overflow          = 50 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.max_pool_size         = 5 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117584 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.max_retries           = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.117987 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.mysql_enable_ndb      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.126109 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.lockutils [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] Lock "cleanup_incomplete_backups_133857" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.001s {{(pid=134303) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}}
Nov 20 20:37:28.126704 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] Cleaning up incomplete backup operations.
Nov 20 20:37:28.127069 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.127516 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.pool_timeout          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.127898 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.retry_interval        = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.128287 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.slave_connection      = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.128656 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.sqlite_synchronous    = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.129015 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] database.use_db_reconnect      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.129410 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.129763 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.138213 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.138213 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.138213 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.138740 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.connection_string     = messaging:// {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.138979 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.enabled               = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.139337 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.es_doc_type           = notification {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.139711 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.es_scroll_size        = 10000 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.140062 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.es_scroll_time        = 2m {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.140420 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.filter_error_trace    = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.140786 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.141166 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.sentinel_service_name = mymaster {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.141521 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.socket_timeout        = 0.1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.141876 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] profiler.trace_sqlalchemy      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.142166 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 20:37:28.153688 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] privsep_osbrick.capabilities   = [21] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.154167 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] privsep_osbrick.group          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.154479 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] privsep_osbrick.helper_command = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.154880 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] privsep_osbrick.logger_name    = oslo_privsep.daemon {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.155247 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.155859 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] privsep_osbrick.user           = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.156257 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_sys_admin.capabilities  = [0, 1, 2, 3, 12, 21] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.164317 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_sys_admin.group         = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.164317 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_sys_admin.helper_command = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.164548 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_sys_admin.logger_name   = oslo_privsep.daemon {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.164915 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.165273 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] cinder_sys_admin.user          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.165634 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.auth_section      = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.166013 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.auth_type         = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.166547 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.cafile            = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.166893 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.certfile          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.167273 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.collect_timing    = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.167621 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.insecure          = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.167964 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.keyfile           = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.168337 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.send_service_user_token = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.168682 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.split_loggers     = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.178694 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] service_user.timeout           = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.179192 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.backend_availability_zone = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.179590 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.chap_password = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.179968 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.chap_username =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.180346 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.180728 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.181092 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.181447 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.driver_client_cert = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.188718 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.driver_client_cert_key = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.189794 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.driver_data_namespace = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.191213 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.192318 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.193430 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.driver_use_ssl = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.194444 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.enable_deferred_deletion = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.195364 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.enable_unsupported_driver = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.196301 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.197200 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.filter_function = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.199337 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.goodness_function = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.200400 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iet_conf      = /etc/iet/ietd.conf {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.201616 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.202705 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.203514 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.204328 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.204930 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.205522 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iscsi_iotype  = fileio {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.206132 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.206701 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iscsi_target_flags =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.207285 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iscsi_write_cache = on {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.207867 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iser_helper   = tgtadm {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.209036 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iser_ip_address = 10.222.0.41 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.209729 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iser_port     = 3260 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.210340 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.210929 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.211684 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.212360 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.num_shell_tries = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.212918 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.213641 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.nvmet_ns_id   = 10 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.214324 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.nvmet_port_id = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.214891 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.215445 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rados_connection_interval = 5 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.216101 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rados_connection_retries = 3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.216677 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_ceph_conf =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.217253 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.217813 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.218393 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.219162 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.219744 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_pool      = rbd {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.220308 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_secret_uuid = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.220858 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.221612 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.rbd_user      = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.222609 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.223193 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.replication_device = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.223864 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.report_discard_supported = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.224442 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.225006 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.reserved_percentage = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.225570 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.scst_target_driver = iscsi {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.226155 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.scst_target_iqn_name = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.226723 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.storage_protocol = iscsi {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.227285 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.target_helper = tgtadm {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.227980 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.target_ip_address = 10.222.0.41 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.228543 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.target_port   = 3260 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.229566 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.230702 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.target_protocol = iscsi {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.231283 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.trace_flags   = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.232000 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.unique_fqdn_network = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.232773 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.use_chap_auth = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.233338 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.234018 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_backend_name = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.234607 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_clear  = zero {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.235167 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_clear_ionice = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.235725 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_clear_size = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.236307 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.236893 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.237472 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.238165 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] backend_defaults.volumes_dir   = /opt/stack/data/cinder/volumes {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.238776 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_reports.file_event_handler = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.239573 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.240171 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_reports.log_dir           = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.241051 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.241880 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.242592 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.243252 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.243986 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.245137 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.245728 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.246320 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.246893 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.247460 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.248056 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.248627 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.249377 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.250032 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.250641 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.251229 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.251911 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.252475 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.253211 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.253786 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.254501 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.255120 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.ssl      = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.255697 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.256261 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.256831 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.257428 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.258303 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_notifications.driver = [] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.258919 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_notifications.retry = -1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.259956 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.261045 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] oslo_messaging_notifications.transport_url = **** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.261686 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] key_manager.backend            = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.262437 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.263241 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.barbican_api_version  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.263802 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.barbican_endpoint     = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.264537 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.barbican_endpoint_type = public {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.265088 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.barbican_region_name  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.265662 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.cafile                = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.266231 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.certfile              = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.266815 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.collect_timing        = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.267619 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.insecure              = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.268184 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.keyfile               = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.268763 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.number_of_retries     = 60 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.270200 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.retry_delay           = 1 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.270674 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.send_service_user_token = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.271129 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.split_loggers         = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.271580 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.timeout               = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.verify_ssl            = True {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican.verify_ssl_path       = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.auth_section = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.auth_type = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.cafile   = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.certfile = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.276872 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.collect_timing = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.277406 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.insecure = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.277406 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.keyfile  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.277406 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.split_loggers = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.277406 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] barbican_service_user.timeout  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.277406 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.approle_role_id          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.278104 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.approle_secret_id        = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.278536 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.cafile                   = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.279110 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.certfile                 = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.279565 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.collect_timing           = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.280234 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.insecure                 = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.280938 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.keyfile                  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.281408 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.kv_mountpoint            = secret {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.282166 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.kv_version               = 2 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.282684 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.namespace                = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.283443 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.root_token_id            = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.283896 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.split_loggers            = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.284342 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.ssl_ca_crt_file          = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.284920 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.timeout                  = None {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.285376 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.use_ssl                  = False {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.285856 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Nov 20 20:37:28.286266 ncdv-master-1 cinder-backup[133857]: DEBUG oslo_service.service [None req-ea25c1cd-33f5-48ec-8dcd-caba8a05bd54 None None] ******************************************************************************** {{(pid=133857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}}
Nov 20 20:37:28.344396 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.lockutils [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] Lock "cleanup_incomplete_backups_133857" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.219s {{(pid=134303) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}}
Nov 20 20:37:28.352586 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 20:37:28.359030 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:37:28.402328 ncdv-master-1 cinder-backup[134303]: INFO cinder.keymgr.migration [None req-d2092887-af58-43ee-8248-17c3f3b668e6 None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Nov 20 20:37:28.467634 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.service [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] Creating RPC server for service cinder-backup {{(pid=134303) start /opt/stack/cinder/cinder/service.py:231}}
Nov 20 20:37:28.486917 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.service [None req-eb301526-e96f-4ad8-be1b-71209b0f26ce None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=134303) start /opt/stack/cinder/cinder/service.py:237}}
Nov 20 20:38:54.551121 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:38:54.551741 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:39:54.548277 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:39:54.548820 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:40:54.546746 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:40:54.547312 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:41:54.551483 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:41:54.552012 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:42:54.555969 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:42:54.556507 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:43:54.561210 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:43:54.561760 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:44:54.566131 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:44:54.566769 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:45:54.572112 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:45:54.572727 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:46:54.569096 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:46:54.569846 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:47:54.570931 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:47:54.571694 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:48:54.575897 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:48:54.576599 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:49:54.577394 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:49:54.578275 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:50:54.575002 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:50:54.575719 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:51:54.575390 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:51:54.576056 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:52:54.573706 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:52:54.574440 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:53:54.570064 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:53:54.571098 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:54:54.568364 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:54:54.569060 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:55:54.570591 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:55:54.571202 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:56:54.569181 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:56:54.569884 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:57:54.569520 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:57:54.570572 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:58:54.568260 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:58:54.569810 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 20:59:54.565301 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 20:59:54.566929 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:00:54.562889 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:00:54.563551 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:01:54.568471 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:01:54.569075 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:02:54.571964 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:02:54.572721 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:03:54.569959 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:03:54.570669 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:04:54.568163 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:04:54.569192 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:05:54.568899 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:05:54.569600 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:06:54.569203 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:06:54.570837 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:07:54.568079 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:07:54.568821 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:08:54.564710 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:08:54.565373 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:09:54.570078 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:09:54.570670 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:10:41.866322 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Create backup started, backup: 98b2fc72-ce3f-4a30-a005-b63af66fd059 volume: 9caac17a-9de9-446c-8f80-5ea71b8d2ccc snapshot: 3a9a6f32-0e20-452b-8e50-df9f4cf8e2e0.
Nov 20 21:10:41.881574 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:10:41.883627 ncdv-master-1 cinder-backup[134303]: INFO oslo.privsep.daemon [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpkunuz66k/privsep.sock']
Nov 20 21:10:42.011949 ncdv-master-1 sudo[170147]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpkunuz66k/privsep.sock
Nov 20 21:10:42.025443 ncdv-master-1 sudo[170147]: pam_unix(sudo:session): session opened for user root by (uid=0)
Nov 20 21:10:44.442729 ncdv-master-1 sudo[170147]: pam_unix(sudo:session): session closed for user root
Nov 20 21:10:44.454999 ncdv-master-1 cinder-backup[134303]: INFO oslo.privsep.daemon [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Spawned new privsep daemon via rootwrap
Nov 20 21:10:44.456410 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Accepted privsep connection to /tmp/tmpkunuz66k/privsep.sock {{(pid=134303) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}}
Nov 20 21:10:44.457888 ncdv-master-1 cinder-backup[134303]: INFO oslo.privsep.daemon [-] privsep daemon starting
Nov 20 21:10:44.458883 ncdv-master-1 cinder-backup[134303]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Nov 20 21:10:44.459314 ncdv-master-1 cinder-backup[134303]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Nov 20 21:10:44.459708 ncdv-master-1 cinder-backup[134303]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 170187
Nov 20 21:10:44.461417 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[25af00a6-f52d-4a1e-b585-e83639d1ef0e]: (2,) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.696541 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.727531 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.031s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:10:44.728568 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[19641ab1-b839-4c5a-a0b4-7e13b89a8801]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.730084 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.742682 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c432b072-68d7-4a97-995c-5ebedfd3ed0c]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:10:44.754900 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:10:44.757426 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[c432b072-68d7-4a97-995c-5ebedfd3ed0c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.757426 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.763798 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[cc2ff697-bf47-4edf-97cc-1d9985f2e67a]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:10:44.764213 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:10:44.765440 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc2ff697-bf47-4edf-97cc-1d9985f2e67a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.766586 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.806502 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.040s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:10:44.807020 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[94b92f1b-4b59-44bd-9eb5-338da804e24b]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.808195 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.859791 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.051s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:10:44.860083 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fc953d2-8e20-4ba1-b531-74fe6010b848]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.861460 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.868371 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:10:44.868707 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[653f8cb6-567d-4a14-ba3e-fbdd294a9c2f]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:10:44.869739 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:10:44.900678 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] CMD "nvme version" returned: 0 in 0.031s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:10:44.920465 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] <== get_connector_properties: return (3037ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:10:46.009618 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:10:46.011936 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4cdcd90>, 'connection_properties': {'name': 'volumes/volume-3b9d1ba1-5b72-4a67-81e6-304902ddcc9c', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '3b9d1ba1-5b72-4a67-81e6-304902ddcc9c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:10:46.013213 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:10:46.106381 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] <== connect_volume: return (93ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4c74af0>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:10:46.116132 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:10:46.145783 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Starting backup of volume='9caac17a-9de9-446c-8f80-5ea71b8d2ccc'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:10:46.188085 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:10:46.188762 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Creating backup base image='volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.base' for volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc. {{(pid=134303) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}}
Nov 20 21:10:47.570742 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Copying data from volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc. {{(pid=134303) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}}
Nov 20 21:10:47.589272 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferring data between 'volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc' and 'volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.base' {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}}
Nov 20 21:10:47.589666 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] 8 chunks of 134217728 bytes to be transferred {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}}
Nov 20 21:10:48.630474 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 1 of 8 (126022K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:49.651141 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 2 of 8 (128505K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:50.743507 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 3 of 8 (120129K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:51.758292 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 4 of 8 (129240K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:52.757743 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 5 of 8 (131232K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:53.843395 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 6 of 8 (120792K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:54.938826 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 7 of 8 (119716K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:56.067225 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Transferred chunk 8 of 8 (116236K/s) {{(pid=134303) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Nov 20 21:10:56.113829 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:10:56.130043 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:10:56.131293 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:10:56.146249 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:10:56.147269 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:10:56.157271 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] No metadata type 'volume-glance-metadata' available {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Nov 20 21:10:56.158327 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Backing up metadata for volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:10:56.217491 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Backup '98b2fc72-ce3f-4a30-a005-b63af66fd059' of volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:10:56.218939 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4cdcd90>, 'connection_properties': {'name': 'volumes/volume-3b9d1ba1-5b72-4a67-81e6-304902ddcc9c', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '3b9d1ba1-5b72-4a67-81e6-304902ddcc9c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4c74af0>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:10:56.231174 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] <== disconnect_volume: return (11ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:10:56.331521 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-7724766f-b659-460e-8608-d3a4473c1040 tempest-VolumesSnapshotTestJSON-1178343811 None] Create backup finished. backup: 98b2fc72-ce3f-4a30-a005-b63af66fd059.
Nov 20 21:10:57.289886 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Delete backup started, backup: 98b2fc72-ce3f-4a30-a005-b63af66fd059.
Nov 20 21:10:57.291636 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Delete started for backup=98b2fc72-ce3f-4a30-a005-b63af66fd059 {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:10:57.292166 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Trying diff format basename='volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.98b2fc72-ce3f-4a30-a005-b63af66fd059' for backup base image of volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:10:57.325641 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Image 'volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.98b2fc72-ce3f-4a30-a005-b63af66fd059' not found - trying diff format name {{(pid=134303) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}}
Nov 20 21:10:57.346798 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Looking for snapshot of backup base 'volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.base' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:10:57.347182 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Backup base 'volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.base' has no snapshots {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}}
Nov 20 21:10:57.347842 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] No backup snapshot to delete {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Nov 20 21:10:57.354248 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Deleting backup base image='volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.base' of volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc.
Nov 20 21:10:57.931721 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Base backup image='volume-9caac17a-9de9-446c-8f80-5ea71b8d2ccc.backup.base' of volume 9caac17a-9de9-446c-8f80-5ea71b8d2ccc deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:10:57.971349 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Delete of backup '98b2fc72-ce3f-4a30-a005-b63af66fd059' for volume '9caac17a-9de9-446c-8f80-5ea71b8d2ccc' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:10:57.978438 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:10:58.002503 ncdv-master-1 cinder-backup[134303]: WARNING cinder.quota [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Deprecated: Default quota for resource: backups is set by the default quota flag: quota_backups, it is now deprecated. Please use the default quota class for default quota.
Nov 20 21:10:58.003411 ncdv-master-1 cinder-backup[134303]: WARNING cinder.quota [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Deprecated: Default quota for resource: backup_gigabytes is set by the default quota flag: quota_backup_gigabytes, it is now deprecated. Please use the default quota class for default quota.
Nov 20 21:10:58.023811 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Created reservations ['961018c2-da05-4ebd-91ea-0c4516a1004f', '282462ba-d9a2-4e83-b667-e165982b0996'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:10:58.067637 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-88317d1c-ebfa-4807-a53c-566e8aec340a tempest-VolumesSnapshotTestJSON-1178343811 None] Delete backup finished, backup 98b2fc72-ce3f-4a30-a005-b63af66fd059 deleted.
Nov 20 21:11:54.579796 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:11:54.580399 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:12:54.584788 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:12:54.585443 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:13:54.589772 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:13:54.590524 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:14:52.969183 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Create backup started, backup: 71496ad9-9d3b-4d2a-8169-6aa64504c878 volume: cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.
Nov 20 21:14:52.980884 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:14:52.982713 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.006597 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.024s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:14:53.007749 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d204e09-7ef6-4437-916b-d67792a037af]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:14:53.010733 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.027147 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f4bbe0d4-3b5f-43a9-b7a8-54918837443a]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:14:53.027666 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:14:53.029539 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4bbe0d4-3b5f-43a9-b7a8-54918837443a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:14:53.032953 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.050846 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f4e51cc2-c8ab-4ec9-afda-df09e7fca0da]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:14:53.051399 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:14:53.053386 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4e51cc2-c8ab-4ec9-afda-df09e7fca0da]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:14:53.057045 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.080837 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.025s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:14:53.081423 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[82b089cd-4151-477c-b0b2-08191c964a58]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:14:53.083671 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.122557 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.039s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:14:53.124041 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e027793-7ecc-4892-902b-f39f369e3366]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:14:53.126646 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.158032 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.033s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:14:53.159095 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[b453173a-9982-4ac9-a056-26f63206e971]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:14:53.159965 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:14:53.196431 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] CMD "nvme version" returned: 0 in 0.036s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:14:53.197986 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] <== get_connector_properties: return (215ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:14:54.595650 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:14:54.596650 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:14:54.645162 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:14:54.647119 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d439d850>, 'connection_properties': {'name': 'volumes/volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:14:54.648438 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:14:54.762709 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] <== connect_volume: return (114ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4329ac0>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:14:54.776077 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:14:54.801637 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Starting backup of volume='cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:14:54.804540 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Volume file is RBD: attempting optimized backup {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Nov 20 21:14:54.879346 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Creating base image 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' {{(pid=134303) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Nov 20 21:14:54.879836 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:14:54.953326 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Using --from-snap 'None' for incremental backup of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4. {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Nov 20 21:14:54.953738 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Creating backup snapshot='backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503' {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Nov 20 21:14:55.740720 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Performing differential transfer from 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4' to 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:14:55.741817 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_f0pakta6 --pool volumes volumes/volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4@backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:14:55.742423 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:14:56.807835 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Differential backup transfer completed in 1.0682s {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Nov 20 21:14:56.808489 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:14:56.829366 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:14:56.829786 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:14:56.842966 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:14:56.843330 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:14:56.866988 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] No metadata type 'volume-glance-metadata' available {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Nov 20 21:14:56.867927 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Backing up metadata for volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:14:56.940845 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Backup '71496ad9-9d3b-4d2a-8169-6aa64504c878' of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4 finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:14:56.943195 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d439d850>, 'connection_properties': {'name': 'volumes/volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4329ac0>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:14:56.970253 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] <== disconnect_volume: return (22ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:14:57.061804 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-5cb8bd10-56c9-47ea-8c34-4de7aaf6af38 tempest-VolumesBackupsAdminTest-848605363 None] Create backup finished. backup: 71496ad9-9d3b-4d2a-8169-6aa64504c878.
Nov 20 21:14:57.701017 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-8371d0ba-4716-4d91-ba22-dab83d766107 tempest-VolumesBackupsAdminTest-805673867 None] Export record started, backup: 71496ad9-9d3b-4d2a-8169-6aa64504c878.
Nov 20 21:14:57.703765 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-8371d0ba-4716-4d91-ba22-dab83d766107 tempest-VolumesBackupsAdminTest-805673867 None] Export record finished, backup 71496ad9-9d3b-4d2a-8169-6aa64504c878 exported.
Nov 20 21:14:58.014492 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-3963643f-5fa5-4601-8ef3-8e615ba6c3a5 tempest-VolumesBackupsAdminTest-805673867 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiMDdiMWNiMzMtNzQyNS00MGQ4LWE2NmMtMmE5NGM0ZmMwMzI5IiwgInVzZXJfaWQiOiAiYjJjZTJmNmMzYWZmNDZkYTk3ZGI3OTc3MmRmYmI0ZTQiLCAicHJvamVjdF9pZCI6ICJlODQwNzI3ZjVkYjE0YjgxODc2OWY0YzRkYWNmNTg1NCIsICJ2b2x1bWVfaWQiOiAiY2RmMGI1NDItYWRjNC00YWQ0LWI4YTUtZjMxZWYyNWVjOGE0IiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC0yMDg3NjcwODk2IiwgImRpc3BsYXlfZGVzY3JpcHRpb24iOiBudWxsLCAic2VydmljZV9tZXRhZGF0YSI6ICJ7XCJiYXNlXCI6IFwidm9sdW1lLWNkZjBiNTQyLWFkYzQtNGFkNC1iOGE1LWYzMWVmMjVlYzhhNC5iYWNrdXAuNzE0OTZhZDktOWQzYi00ZDJhLTgxNjktNmFhNjQ1MDRjODc4XCJ9IiwgInNlcnZpY2UiOiAiY2luZGVyLmJhY2t1cC5kcml2ZXJzLmNlcGguQ2VwaEJhY2t1cERyaXZlciIsICJvYmplY3RfY291bnQiOiAwLCAidGVtcF92b2x1bWVfaWQiOiBudWxsLCAidGVtcF9zbmFwc2hvdF9pZCI6IG51bGwsICJudW1fZGVwZW5kZW50X2JhY2t1cHMiOiAwLCAic25hcHNob3RfaWQiOiBudWxsLCAiZGF0YV90aW1lc3RhbXAiOiAiMjAyMS0xMS0yMFQyMToxNDo1M1oiLCAicmVzdG9yZV92b2x1bWVfaWQiOiBudWxsLCAibWV0YWRhdGEiOiB7fSwgImVuY3J5cHRpb25fa2V5X2lkIjogbnVsbCwgImNyZWF0ZWRfYXQiOiAiMjAyMS0xMS0yMFQyMToxNDo1M1oiLCAidXBkYXRlZF9hdCI6ICIyMDIxLTExLTIwVDIxOjE0OjU3WiIsICJkZWxldGVkX2F0IjogbnVsbCwgImRlbGV0ZWQiOiBmYWxzZX0=.
Nov 20 21:14:58.032487 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-3963643f-5fa5-4601-8ef3-8e615ba6c3a5 tempest-VolumesBackupsAdminTest-805673867 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:14:58.158093 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-3963643f-5fa5-4601-8ef3-8e615ba6c3a5 tempest-VolumesBackupsAdminTest-805673867 None] Import record id 07b1cb33-7425-40d8-a66c-2a94c4fc0329 metadata from driver finished.
Nov 20 21:15:00.689337 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Restore backup started, backup: 71496ad9-9d3b-4d2a-8169-6aa64504c878 volume: eb35ab6f-4427-44bb-afc6-132727ac2e52.
Nov 20 21:15:00.723169 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:15:00.724706 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.738636 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:00.740987 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0e82d5e-3893-493a-8e47-be9928176599]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:00.742325 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.757563 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3a1f3f94-c8a9-4bb9-8107-317b763c8c0b]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:15:00.758800 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:15:00.760235 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a1f3f94-c8a9-4bb9-8107-317b763c8c0b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:00.760948 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.768969 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a598cd33-e049-4117-a249-1592f0753280]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:15:00.770382 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:15:00.771831 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[a598cd33-e049-4117-a249-1592f0753280]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:00.772526 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.785256 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.012s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:00.785621 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[a753ec22-843f-4069-9c07-d65f2983e2af]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:00.788345 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.808433 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.021s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:00.809555 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[d18c19fe-6267-4ca4-b305-69c8ce7a99b3]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:00.812164 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.823890 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.011s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:00.826470 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[bdd4885d-bdc6-4e40-ba51-e8eb3d08193c]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:00.827630 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:00.855552 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] CMD "nvme version" returned: 0 in 0.028s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:00.857058 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:15:01.595352 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:15:01.596708 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d43957f0>, 'connection_properties': {'name': 'volumes/volume-eb35ab6f-4427-44bb-afc6-132727ac2e52', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'eb35ab6f-4427-44bb-afc6-132727ac2e52', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:15:01.597894 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:15:01.672891 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] <== connect_volume: return (75ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42f8790>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:15:01.683945 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:15:01.697315 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Starting restore from Ceph backup=71496ad9-9d3b-4d2a-8169-6aa64504c878 to volume=volume-eb35ab6f-4427-44bb-afc6-132727ac2e52 {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Nov 20 21:15:01.823836 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Looking for snapshot of backup base 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:15:01.825837 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Found snapshot 'backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:15:01.836702 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Volume file is RBD. {{(pid=134303) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Nov 20 21:15:01.910737 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Attempting differential restore. {{(pid=134303) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Nov 20 21:15:01.911219 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Attempting incremental restore from base='volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' snap='backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503' {{(pid=134303) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Nov 20 21:15:01.911606 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Performing differential transfer from 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' to 'volume-eb35ab6f-4427-44bb-afc6-132727ac2e52' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:15:01.912018 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878@backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:15:01.912369 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_ksiean_q --pool volumes - volumes/volume-eb35ab6f-4427-44bb-afc6-132727ac2e52' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:15:02.989504 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Restore transfer completed in 1.0773s {{(pid=134303) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Nov 20 21:15:02.991201 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Nov 20 21:15:03.023081 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Restoring volume base metadata {{(pid=134303) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Nov 20 21:15:03.049043 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] No metadata of type 'volume-metadata' to restore {{(pid=134303) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Nov 20 21:15:03.049757 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=134303) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Nov 20 21:15:03.053820 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Restore to volume eb35ab6f-4427-44bb-afc6-132727ac2e52 finished successfully. {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Nov 20 21:15:03.055606 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d43957f0>, 'connection_properties': {'name': 'volumes/volume-eb35ab6f-4427-44bb-afc6-132727ac2e52', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'eb35ab6f-4427-44bb-afc6-132727ac2e52', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42f8790>}, 'force': True, 'ignore_errors': False}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:15:03.067349 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] <== disconnect_volume: return (10ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:15:03.123372 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.manager [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Encryption key ID for volume eb35ab6f-4427-44bb-afc6-132727ac2e52 already matches encryption key ID in backup 71496ad9-9d3b-4d2a-8169-6aa64504c878. {{(pid=134303) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Nov 20 21:15:03.172101 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-17d21849-f5f8-4feb-a76e-32aaf18fbf73 tempest-VolumesBackupsAdminTest-848605363 None] Finished restoring backup 71496ad9-9d3b-4d2a-8169-6aa64504c878 to volume eb35ab6f-4427-44bb-afc6-132727ac2e52.
Nov 20 21:15:04.179966 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Delete backup started, backup: 07b1cb33-7425-40d8-a66c-2a94c4fc0329.
Nov 20 21:15:04.182274 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Delete started for backup=07b1cb33-7425-40d8-a66c-2a94c4fc0329 {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:15:04.182696 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Trying diff format basename='volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' for backup base image of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:15:04.267331 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Looking for snapshot of backup base 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:15:04.267965 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Backup '07b1cb33-7425-40d8-a66c-2a94c4fc0329' has no snapshot {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}}
Nov 20 21:15:04.268379 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] No backup snapshot to delete {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Nov 20 21:15:04.283515 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Backup base image of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4 still has 1 snapshots so skipping base image delete.
Nov 20 21:15:04.348329 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Metadata backup object 'backup.07b1cb33-7425-40d8-a66c-2a94c4fc0329.meta' not found - ignoring {{(pid=134303) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}}
Nov 20 21:15:04.359891 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Delete of backup '07b1cb33-7425-40d8-a66c-2a94c4fc0329' for volume 'cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:15:04.368362 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:15:04.397837 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Created reservations ['c0ee8529-37c4-42b7-829a-16a9b9b5b14f', '87d6c3e6-ec9b-4904-93fe-3c23873eda6e'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:15:04.431096 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-f6617580-1ed9-44ad-abff-8586cbe73558 tempest-VolumesBackupsAdminTest-805673867 None] Delete backup finished, backup 07b1cb33-7425-40d8-a66c-2a94c4fc0329 deleted.
Nov 20 21:15:05.332687 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Delete backup started, backup: 71496ad9-9d3b-4d2a-8169-6aa64504c878.
Nov 20 21:15:05.335273 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Delete started for backup=71496ad9-9d3b-4d2a-8169-6aa64504c878 {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:15:05.337228 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Trying diff format basename='volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' for backup base image of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:15:05.423924 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Looking for snapshot of backup base 'volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:15:05.424407 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Found snapshot 'backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:15:05.425423 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Deleting backup snapshot='backup.71496ad9-9d3b-4d2a-8169-6aa64504c878.snap.1637442894.9533503' {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Nov 20 21:15:06.020006 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Deleting backup base image='volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.
Nov 20 21:15:06.245577 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Base backup image='volume-cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4.backup.71496ad9-9d3b-4d2a-8169-6aa64504c878' of volume cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4 deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:15:06.286023 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Delete of backup '71496ad9-9d3b-4d2a-8169-6aa64504c878' for volume 'cdf0b542-adc4-4ad4-b8a5-f31ef25ec8a4' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:15:06.291742 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:15:06.319854 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Created reservations ['43ff888c-2665-43ba-b6a8-2d8c1789c7fc', 'e1f925ca-1b39-4ac4-9a2e-12028bcafa9d'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:15:06.346970 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-0e58ed4d-9c99-4edf-a925-05aad1a7e457 tempest-VolumesBackupsAdminTest-848605363 None] Delete backup finished, backup 71496ad9-9d3b-4d2a-8169-6aa64504c878 deleted.
Nov 20 21:15:07.997208 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Create backup started, backup: 0a3ee2ae-e7ac-4326-b38d-87c0677871a3 volume: 2c5ebbcb-4d38-4a44-9a19-ea90342400f6.
Nov 20 21:15:08.008751 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:15:08.011019 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.028032 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:08.029278 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ded24ff-bf62-4d15-9e96-72aaa5635532]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:08.032662 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.044443 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0b858d10-b379-437b-bcba-6c5d73adc682]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:15:08.046534 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:15:08.048274 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b858d10-b379-437b-bcba-6c5d73adc682]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:08.050340 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.065818 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e7db6aa0-875b-4368-9355-181d84717202]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:15:08.066363 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:15:08.067320 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7db6aa0-875b-4368-9355-181d84717202]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:08.068603 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.107611 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.039s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:08.108363 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e31d190-a443-4c7a-a1ea-e1b9182ef17d]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:08.109854 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.138603 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.028s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:08.142303 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc93d007-498c-4aa9-9129-40c133376316]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:08.147285 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.183849 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.039s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:08.184188 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e66f89f-c031-48d9-8d79-a74d11b40ef4]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:15:08.185194 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:15:08.227476 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] CMD "nvme version" returned: 0 in 0.041s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:15:08.228602 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] <== get_connector_properties: return (218ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:15:08.913209 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-d107c424-2835-4e4a-84b8-c0534fb1e9e0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:15:09.621080 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:15:09.623123 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4da71f0>, 'connection_properties': {'name': 'volumes/volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '2c5ebbcb-4d38-4a44-9a19-ea90342400f6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:15:09.624936 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:15:09.754359 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] <== connect_volume: return (130ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d43159a0>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:15:09.781968 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Starting backup of volume='2c5ebbcb-4d38-4a44-9a19-ea90342400f6'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:15:09.786873 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Volume file is RBD: attempting optimized backup {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Nov 20 21:15:09.840388 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Creating base image 'volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' {{(pid=134303) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Nov 20 21:15:09.843122 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:15:09.906760 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Using --from-snap 'None' for incremental backup of volume 2c5ebbcb-4d38-4a44-9a19-ea90342400f6. {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Nov 20 21:15:09.907658 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Creating backup snapshot='backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3.snap.1637442909.9072242' {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Nov 20 21:15:10.618305 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Performing differential transfer from 'volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6' to 'volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:15:10.619162 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_8isv0cki --pool volumes volumes/volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6@backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3.snap.1637442909.9072242 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:15:10.619898 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:15:11.757384 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Differential backup transfer completed in 1.1389s {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Nov 20 21:15:11.758795 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:15:11.776871 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:15:11.777724 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:15:11.786863 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:15:11.787628 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:15:11.799385 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] No metadata type 'volume-glance-metadata' available {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Nov 20 21:15:11.800234 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Backing up metadata for volume 2c5ebbcb-4d38-4a44-9a19-ea90342400f6. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:15:11.911009 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Backup '0a3ee2ae-e7ac-4326-b38d-87c0677871a3' of volume 2c5ebbcb-4d38-4a44-9a19-ea90342400f6 finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:15:11.913892 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4da71f0>, 'connection_properties': {'name': 'volumes/volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '2c5ebbcb-4d38-4a44-9a19-ea90342400f6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d43159a0>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:15:11.954286 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] <== disconnect_volume: return (38ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:15:12.063890 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-222365dd-d095-4c52-9056-8934c267a811 tempest-VolumesBackupsAdminTest-848605363 None] Create backup finished. backup: 0a3ee2ae-e7ac-4326-b38d-87c0677871a3.
Nov 20 21:15:12.339988 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-5e3af837-ba63-40ad-a619-3404ebbe0a48 tempest-VolumesBackupsAdminTest-805673867 None] Reset backup status started, backup_id: 0a3ee2ae-e7ac-4326-b38d-87c0677871a3, status: error.
Nov 20 21:15:12.340523 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-5e3af837-ba63-40ad-a619-3404ebbe0a48 tempest-VolumesBackupsAdminTest-805673867 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver.
Nov 20 21:15:12.424248 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Delete backup started, backup: 0a3ee2ae-e7ac-4326-b38d-87c0677871a3.
Nov 20 21:15:12.426889 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Delete started for backup=0a3ee2ae-e7ac-4326-b38d-87c0677871a3 {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:15:12.427316 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Trying diff format basename='volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' for backup base image of volume 2c5ebbcb-4d38-4a44-9a19-ea90342400f6. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:15:12.558816 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Looking for snapshot of backup base 'volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:15:12.559357 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Found snapshot 'backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3.snap.1637442909.9072242' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:15:12.560106 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Deleting backup snapshot='backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3.snap.1637442909.9072242' {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Nov 20 21:15:13.644568 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Deleting backup base image='volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' of volume 2c5ebbcb-4d38-4a44-9a19-ea90342400f6.
Nov 20 21:15:14.024464 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Base backup image='volume-2c5ebbcb-4d38-4a44-9a19-ea90342400f6.backup.0a3ee2ae-e7ac-4326-b38d-87c0677871a3' of volume 2c5ebbcb-4d38-4a44-9a19-ea90342400f6 deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:15:14.107892 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Delete of backup '0a3ee2ae-e7ac-4326-b38d-87c0677871a3' for volume '2c5ebbcb-4d38-4a44-9a19-ea90342400f6' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:15:14.113994 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:15:14.170358 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Created reservations ['a9627386-b82f-471a-acd8-5569c4e3d875', '355ebf65-8753-47e4-bb9e-289edb8faf05'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:15:14.212147 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-bb31648c-2e28-4f79-8057-920b8d91833e tempest-VolumesBackupsAdminTest-848605363 None] Delete backup finished, backup 0a3ee2ae-e7ac-4326-b38d-87c0677871a3 deleted.
Nov 20 21:15:54.597812 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:15:54.598540 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:16:54.603493 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:16:54.604215 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:17:11.227246 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Create backup started, backup: dda0613a-e4a9-4210-b0c1-418b090a1872 volume: 7db9043d-ea67-4d40-b95e-09a5d3335c47.
Nov 20 21:17:11.238167 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:11.240583 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.250444 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.010s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:11.251159 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[43052015-6d38-4ac0-846e-88d34f1e23cb]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:11.253491 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.262178 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a745ce03-dfd1-4aaf-9a09-a6655cec1c22]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:17:11.263284 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:17:11.264508 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[a745ce03-dfd1-4aaf-9a09-a6655cec1c22]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:11.265568 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.272514 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[7fa46c99-c5f2-4f87-8cc2-8350e67fe9f9]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:17:11.273601 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:17:11.275427 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fa46c99-c5f2-4f87-8cc2-8350e67fe9f9]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:11.279120 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.309273 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.032s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:11.309870 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[39c09649-c189-4f06-b428-b805206f099d]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:11.311157 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.329092 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.017s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:11.329659 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0bf124f-8fc0-45a6-b194-00640c76bf21]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:11.333339 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.339623 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:11.340502 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc49052e-c51b-4217-aa1b-a71f34440159]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:11.341308 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:11.390002 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] CMD "nvme version" returned: 0 in 0.047s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:11.391922 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] <== get_connector_properties: return (152ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:12.250603 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:17:12.252454 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4329940>, 'connection_properties': {'name': 'volumes/volume-7db9043d-ea67-4d40-b95e-09a5d3335c47', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7db9043d-ea67-4d40-b95e-09a5d3335c47', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:12.253844 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:17:12.310760 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] <== connect_volume: return (57ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4317d60>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:12.319516 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:17:12.341108 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Starting backup of volume='7db9043d-ea67-4d40-b95e-09a5d3335c47'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:17:12.342343 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Volume file is RBD: attempting optimized backup {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Nov 20 21:17:12.371081 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Creating base image 'volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' {{(pid=134303) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Nov 20 21:17:12.371686 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:17:12.420270 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Using --from-snap 'None' for incremental backup of volume 7db9043d-ea67-4d40-b95e-09a5d3335c47. {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Nov 20 21:17:12.420797 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Creating backup snapshot='backup.dda0613a-e4a9-4210-b0c1-418b090a1872.snap.1637443032.420295' {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Nov 20 21:17:12.881584 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Performing differential transfer from 'volume-7db9043d-ea67-4d40-b95e-09a5d3335c47' to 'volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:17:12.882000 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_x5lkcy2f --pool volumes volumes/volume-7db9043d-ea67-4d40-b95e-09a5d3335c47@backup.dda0613a-e4a9-4210-b0c1-418b090a1872.snap.1637443032.420295 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:17:12.882445 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:17:13.929868 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Differential backup transfer completed in 1.0478s {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Nov 20 21:17:13.930914 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:17:13.947429 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:17:13.947961 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:17:13.955981 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:17:13.956519 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:17:13.964483 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] No metadata type 'volume-glance-metadata' available {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Nov 20 21:17:13.965919 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Backing up metadata for volume 7db9043d-ea67-4d40-b95e-09a5d3335c47. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:17:14.001591 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Backup 'dda0613a-e4a9-4210-b0c1-418b090a1872' of volume 7db9043d-ea67-4d40-b95e-09a5d3335c47 finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:17:14.003261 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4329940>, 'connection_properties': {'name': 'volumes/volume-7db9043d-ea67-4d40-b95e-09a5d3335c47', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7db9043d-ea67-4d40-b95e-09a5d3335c47', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4317d60>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:14.014505 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] <== disconnect_volume: return (10ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:14.084436 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-62ce7bac-e971-4296-a961-f41b2c7c48ba tempest-VolumesBackupsTest-964287589 None] Create backup finished. backup: dda0613a-e4a9-4210-b0c1-418b090a1872.
Nov 20 21:17:14.392131 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Delete backup started, backup: dda0613a-e4a9-4210-b0c1-418b090a1872.
Nov 20 21:17:14.394030 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Delete started for backup=dda0613a-e4a9-4210-b0c1-418b090a1872 {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:17:14.394724 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Trying diff format basename='volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' for backup base image of volume 7db9043d-ea67-4d40-b95e-09a5d3335c47. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:17:14.480962 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Looking for snapshot of backup base 'volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:17:14.481387 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Found snapshot 'backup.dda0613a-e4a9-4210-b0c1-418b090a1872.snap.1637443032.420295' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:17:14.481741 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Deleting backup snapshot='backup.dda0613a-e4a9-4210-b0c1-418b090a1872.snap.1637443032.420295' {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Nov 20 21:17:14.925326 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Deleting backup base image='volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' of volume 7db9043d-ea67-4d40-b95e-09a5d3335c47.
Nov 20 21:17:15.208020 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Base backup image='volume-7db9043d-ea67-4d40-b95e-09a5d3335c47.backup.dda0613a-e4a9-4210-b0c1-418b090a1872' of volume 7db9043d-ea67-4d40-b95e-09a5d3335c47 deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:17:15.258790 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Delete of backup 'dda0613a-e4a9-4210-b0c1-418b090a1872' for volume '7db9043d-ea67-4d40-b95e-09a5d3335c47' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:17:15.265465 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:17:15.290092 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Created reservations ['b0e3fdee-0580-4bd6-8e45-1388f8f7f9b0', 'e8971361-07ec-4e20-a4cc-598821f811f6'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:17:15.326702 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-a60b1548-3818-4eb9-a4c9-6081e8fd5115 tempest-VolumesBackupsTest-964287589 None] Delete backup finished, backup dda0613a-e4a9-4210-b0c1-418b090a1872 deleted.
Nov 20 21:17:25.750517 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Create backup started, backup: 2284c44a-c268-439c-ba75-802eae09a4da volume: 63f35122-865d-4f8c-aea6-9339ec61e273.
Nov 20 21:17:25.773658 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:25.777440 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.789094 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:25.789424 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4395fa1-7ad8-4589-894c-5e078566bc5a]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:25.790856 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.799746 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d8f85667-9d02-47fe-89fc-7a8c73c42733]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:17:25.800710 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:17:25.801825 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8f85667-9d02-47fe-89fc-7a8c73c42733]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:25.804185 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.822835 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[30fe8a1a-2d58-4bd1-8238-8ffa65f12ce9]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:17:25.823799 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:17:25.826902 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[30fe8a1a-2d58-4bd1-8238-8ffa65f12ce9]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:25.828819 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.847795 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.019s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:25.848084 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea0f0a97-7cbf-4c89-bde6-c08fa9977cbf]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:25.849994 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.867539 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.017s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:25.868165 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5bc613c-47ea-421b-880f-3ce188502fdb]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:25.873512 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.882376 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.012s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:25.882720 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[fdc4727a-12f6-4bfc-a0e8-ee60f5c62181]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:25.883514 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:25.906845 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] CMD "nvme version" returned: 0 in 0.023s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:25.908704 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] <== get_connector_properties: return (133ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:26.672912 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:17:26.674322 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4c74100>, 'connection_properties': {'name': 'volumes/volume-63f35122-865d-4f8c-aea6-9339ec61e273', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '63f35122-865d-4f8c-aea6-9339ec61e273', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:26.677389 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:17:26.789955 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] <== connect_volume: return (114ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4330c40>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:26.798783 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:17:26.823212 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Starting backup of volume='63f35122-865d-4f8c-aea6-9339ec61e273'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:17:26.824157 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Volume file is RBD: attempting optimized backup {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Nov 20 21:17:26.851168 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Creating base image 'volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' {{(pid=134303) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Nov 20 21:17:26.851715 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:17:26.930232 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Using --from-snap 'None' for incremental backup of volume 63f35122-865d-4f8c-aea6-9339ec61e273. {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Nov 20 21:17:26.930936 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Creating backup snapshot='backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573' {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Nov 20 21:17:27.814154 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Performing differential transfer from 'volume-63f35122-865d-4f8c-aea6-9339ec61e273' to 'volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:17:27.815300 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_ebimbid_ --pool volumes volumes/volume-63f35122-865d-4f8c-aea6-9339ec61e273@backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:17:27.815679 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:17:34.949261 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Differential backup transfer completed in 7.1341s {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Nov 20 21:17:34.950278 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:17:34.962951 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:17:34.963727 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:17:34.971808 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:17:34.972353 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:17:34.983145 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Nov 20 21:17:34.983725 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Backing up metadata for volume 63f35122-865d-4f8c-aea6-9339ec61e273. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:17:35.059819 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Backup '2284c44a-c268-439c-ba75-802eae09a4da' of volume 63f35122-865d-4f8c-aea6-9339ec61e273 finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:17:35.062966 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4c74100>, 'connection_properties': {'name': 'volumes/volume-63f35122-865d-4f8c-aea6-9339ec61e273', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '63f35122-865d-4f8c-aea6-9339ec61e273', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4330c40>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:35.096013 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] <== disconnect_volume: return (31ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:35.168750 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-27bd7145-3d7d-433c-9df5-f6c6856cb4d8 tempest-VolumesBackupsTest-964287589 None] Create backup finished. backup: 2284c44a-c268-439c-ba75-802eae09a4da.
Nov 20 21:17:37.506859 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Restore backup started, backup: 2284c44a-c268-439c-ba75-802eae09a4da volume: 4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1.
Nov 20 21:17:37.528541 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:37.533059 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.555130 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.025s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:37.555450 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[c715c928-5207-4116-89b7-4d66465d8267]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:37.557406 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.572504 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0f534124-27ed-4b67-9540-5a55a7f6f151]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:17:37.573385 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:17:37.574812 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f534124-27ed-4b67-9540-5a55a7f6f151]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:37.576278 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.583277 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ba1177af-5f69-4ae7-b6a9-0c39b5420b93]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:17:37.585246 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:17:37.586967 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba1177af-5f69-4ae7-b6a9-0c39b5420b93]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:37.589139 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.615881 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.028s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:37.616580 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[7474ec31-fcc3-414b-a6ee-2a7407de8c73]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:37.621759 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.636456 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.016s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:37.637073 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[a744a73c-7608-4e8d-bf27-1f70848a0380]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:37.644548 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.656239 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.015s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:37.656810 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[30483840-c38f-4fba-82e7-a509d2c03bd4]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:17:37.657703 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:17:37.680193 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] CMD "nvme version" returned: 0 in 0.022s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:17:37.682490 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] <== get_connector_properties: return (152ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:38.521710 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:17:38.523982 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4309a00>, 'connection_properties': {'name': 'volumes/volume-4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:38.525088 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:17:38.582461 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] <== connect_volume: return (58ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42bd7f0>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:38.592680 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:17:38.618548 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Starting restore from Ceph backup=2284c44a-c268-439c-ba75-802eae09a4da to volume=volume-4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1 {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Nov 20 21:17:38.733549 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Looking for snapshot of backup base 'volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:17:38.733935 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Found snapshot 'backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:17:38.739377 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Volume file is RBD. {{(pid=134303) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Nov 20 21:17:38.810704 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Attempting differential restore. {{(pid=134303) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Nov 20 21:17:38.811451 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Attempting incremental restore from base='volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' snap='backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573' {{(pid=134303) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Nov 20 21:17:38.812003 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Performing differential transfer from 'volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' to 'volume-4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:17:38.812631 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da@backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:17:38.812994 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_y_71_4px --pool volumes - volumes/volume-4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:17:43.004510 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Restore transfer completed in 4.1919s {{(pid=134303) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Nov 20 21:17:43.006504 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Nov 20 21:17:43.030774 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Restoring volume base metadata {{(pid=134303) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Nov 20 21:17:43.056246 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] No metadata of type 'volume-metadata' to restore {{(pid=134303) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Nov 20 21:17:43.057808 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Restoring volume glance metadata {{(pid=134303) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Nov 20 21:17:43.249740 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Restore to volume 4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1 finished successfully. {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Nov 20 21:17:43.252067 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4309a00>, 'connection_properties': {'name': 'volumes/volume-4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42bd7f0>}, 'force': True, 'ignore_errors': False}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:17:43.264843 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] <== disconnect_volume: return (11ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:17:43.359289 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.manager [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Encryption key ID for volume 4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1 already matches encryption key ID in backup 2284c44a-c268-439c-ba75-802eae09a4da. {{(pid=134303) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Nov 20 21:17:43.403498 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-439efcf9-756b-4b62-a1df-68ffeb806ebc tempest-VolumesBackupsTest-964287589 None] Finished restoring backup 2284c44a-c268-439c-ba75-802eae09a4da to volume 4c3ee60c-9ec2-4b45-93c7-8f09c20e3df1.
Nov 20 21:17:47.507092 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Delete backup started, backup: 2284c44a-c268-439c-ba75-802eae09a4da.
Nov 20 21:17:47.509320 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Delete started for backup=2284c44a-c268-439c-ba75-802eae09a4da {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:17:47.509911 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Trying diff format basename='volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' for backup base image of volume 63f35122-865d-4f8c-aea6-9339ec61e273. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:17:47.619946 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Looking for snapshot of backup base 'volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:17:47.621100 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Found snapshot 'backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:17:47.621696 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Deleting backup snapshot='backup.2284c44a-c268-439c-ba75-802eae09a4da.snap.1637443046.9302573' {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Nov 20 21:17:48.046511 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Deleting backup base image='volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' of volume 63f35122-865d-4f8c-aea6-9339ec61e273.
Nov 20 21:17:48.610192 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Base backup image='volume-63f35122-865d-4f8c-aea6-9339ec61e273.backup.2284c44a-c268-439c-ba75-802eae09a4da' of volume 63f35122-865d-4f8c-aea6-9339ec61e273 deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:17:48.660834 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Delete of backup '2284c44a-c268-439c-ba75-802eae09a4da' for volume '63f35122-865d-4f8c-aea6-9339ec61e273' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:17:48.669347 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:17:48.705901 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Created reservations ['9d73e0a7-ae3a-4079-ae79-66f4cfb0ad78', 'b9fc7320-3c4c-4df0-ab3c-70a88809a3a0'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:17:48.739323 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-0986793b-4780-470b-a013-4841d0847891 tempest-VolumesBackupsTest-964287589 None] Delete backup finished, backup 2284c44a-c268-439c-ba75-802eae09a4da deleted.
Nov 20 21:17:54.605629 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:17:54.606219 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:18:31.681140 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Create backup started, backup: edf623ef-7225-4148-8e74-88cafffed96f volume: f4b56d10-2e30-443a-b954-4dd7d242c0ad.
Nov 20 21:18:31.691918 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:18:31.693871 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.710899 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:31.711645 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2575152-9074-4478-971a-21b30c0eaec6]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:31.715986 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.720373 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[85c5e805-a171-4317-9ba8-339d1934cda2]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:18:31.721007 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:18:31.722349 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[85c5e805-a171-4317-9ba8-339d1934cda2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:31.723029 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.729977 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[203f4da8-51c2-43ea-8fad-e863a99950d0]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:18:31.730385 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:18:31.731547 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[203f4da8-51c2-43ea-8fad-e863a99950d0]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:31.732574 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.741935 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:31.742308 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[8804dc9d-5eb9-4f40-8eda-ecee0b9feff3]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:31.744674 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.753858 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:31.754795 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[c40c4eea-1eb1-4454-ba43-ab67f365ba59]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:31.756037 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.765725 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:31.766083 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[be71d145-22d5-4265-9154-a62f789ba5f8]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:31.767008 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:31.787521 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] CMD "nvme version" returned: 0 in 0.020s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:31.788666 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] <== get_connector_properties: return (96ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:18:32.578405 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:18:32.579822 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4c2bac0>, 'connection_properties': {'name': 'volumes/volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f4b56d10-2e30-443a-b954-4dd7d242c0ad', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:18:32.580954 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:18:32.631193 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] <== connect_volume: return (51ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42e9460>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:18:32.640608 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:18:32.660820 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Starting backup of volume='f4b56d10-2e30-443a-b954-4dd7d242c0ad'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:18:32.661857 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Volume file is RBD: attempting optimized backup {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Nov 20 21:18:32.686765 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Creating base image 'volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' {{(pid=134303) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Nov 20 21:18:32.687456 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:18:32.748339 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Using --from-snap 'None' for incremental backup of volume f4b56d10-2e30-443a-b954-4dd7d242c0ad. {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Nov 20 21:18:32.748810 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Creating backup snapshot='backup.edf623ef-7225-4148-8e74-88cafffed96f.snap.1637443112.7483652' {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Nov 20 21:18:33.660698 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Performing differential transfer from 'volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad' to 'volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:18:33.661112 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_ttilis5b --pool volumes volumes/volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad@backup.edf623ef-7225-4148-8e74-88cafffed96f.snap.1637443112.7483652 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:18:33.661568 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:18:34.678464 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Differential backup transfer completed in 1.0173s {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Nov 20 21:18:34.679092 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:18:34.693607 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:18:34.694032 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:18:34.700184 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:18:34.700567 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:18:34.706541 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] No metadata type 'volume-glance-metadata' available {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Nov 20 21:18:34.707075 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Backing up metadata for volume f4b56d10-2e30-443a-b954-4dd7d242c0ad. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:18:34.737667 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Backup 'edf623ef-7225-4148-8e74-88cafffed96f' of volume f4b56d10-2e30-443a-b954-4dd7d242c0ad finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:18:34.739247 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4c2bac0>, 'connection_properties': {'name': 'volumes/volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f4b56d10-2e30-443a-b954-4dd7d242c0ad', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42e9460>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:18:34.765251 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] <== disconnect_volume: return (24ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:18:34.820425 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-297a450c-c0d8-41fa-88c0-76a2a36375eb tempest-VolumesBackupsV39Test-1666588577 None] Create backup finished. backup: edf623ef-7225-4148-8e74-88cafffed96f.
Nov 20 21:18:35.929807 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Delete backup started, backup: edf623ef-7225-4148-8e74-88cafffed96f.
Nov 20 21:18:35.931110 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Delete started for backup=edf623ef-7225-4148-8e74-88cafffed96f {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:18:35.931613 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Trying diff format basename='volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' for backup base image of volume f4b56d10-2e30-443a-b954-4dd7d242c0ad. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:18:35.981503 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Looking for snapshot of backup base 'volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:18:35.981994 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Found snapshot 'backup.edf623ef-7225-4148-8e74-88cafffed96f.snap.1637443112.7483652' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:18:35.982396 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Deleting backup snapshot='backup.edf623ef-7225-4148-8e74-88cafffed96f.snap.1637443112.7483652' {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Nov 20 21:18:36.706794 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Deleting backup base image='volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' of volume f4b56d10-2e30-443a-b954-4dd7d242c0ad.
Nov 20 21:18:36.945298 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Base backup image='volume-f4b56d10-2e30-443a-b954-4dd7d242c0ad.backup.edf623ef-7225-4148-8e74-88cafffed96f' of volume f4b56d10-2e30-443a-b954-4dd7d242c0ad deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:18:36.988630 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Delete of backup 'edf623ef-7225-4148-8e74-88cafffed96f' for volume 'f4b56d10-2e30-443a-b954-4dd7d242c0ad' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:18:36.994442 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:18:37.020262 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Created reservations ['df16ebe5-442f-4147-a808-165041c4fe2f', 'd413f995-0303-4a36-ab8a-3e03f6c123d6'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:18:37.046495 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-f53fe88c-14b2-4d7b-bf41-4b8966a013ca tempest-VolumesBackupsV39Test-1666588577 None] Delete backup finished, backup edf623ef-7225-4148-8e74-88cafffed96f deleted.
Nov 20 21:18:54.020271 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Create backup started, backup: e2d330db-5233-464f-a223-c1b9124b8644 volume: 14992743-5006-407a-ba5a-7bd61fae90ce.
Nov 20 21:18:54.027513 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:18:54.029773 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.037913 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:54.038284 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[2de0046e-543e-4f57-aec1-75251bf52ddc]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:54.041489 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.049659 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[077709ef-c198-449c-a7f1-3ab588976335]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:18:54.050826 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:18:54.052025 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[077709ef-c198-449c-a7f1-3ab588976335]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:54.054536 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.058708 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8e6ffaf4-9020-420b-a9fb-fb34757d18c7]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:18:54.060176 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:18:54.061522 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e6ffaf4-9020-420b-a9fb-fb34757d18c7]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:54.062586 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.071216 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:54.071553 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[29e22d7f-0a71-41f2-b1f2-34ebe66161c4]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:54.074854 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.081648 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:54.081980 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec3055f6-abaf-4e84-be80-8a39ed116a00]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:54.084994 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.091213 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:54.091536 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[61b52f7f-275d-4ea6-9b18-7391df21a411]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:18:54.092260 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:18:54.112155 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:18:54.112923 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] <== get_connector_properties: return (84ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:18:54.601038 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:18:54.601691 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:18:54.883806 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:18:54.885374 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d42e9d90>, 'connection_properties': {'name': 'volumes/volume-14992743-5006-407a-ba5a-7bd61fae90ce', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '14992743-5006-407a-ba5a-7bd61fae90ce', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:18:54.886514 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:18:55.006272 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] <== connect_volume: return (120ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42cc670>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:18:55.031959 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:18:55.063384 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Starting backup of volume='14992743-5006-407a-ba5a-7bd61fae90ce'. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Nov 20 21:18:55.064822 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Volume file is RBD: attempting optimized backup {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Nov 20 21:18:55.110293 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Creating base image 'volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' {{(pid=134303) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Nov 20 21:18:55.110876 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134303) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Nov 20 21:18:55.155529 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Using --from-snap 'None' for incremental backup of volume 14992743-5006-407a-ba5a-7bd61fae90ce. {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Nov 20 21:18:55.156181 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Creating backup snapshot='backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436' {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Nov 20 21:18:55.704104 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Performing differential transfer from 'volume-14992743-5006-407a-ba5a-7bd61fae90ce' to 'volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:18:55.704663 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_akaf2zmp --pool volumes volumes/volume-14992743-5006-407a-ba5a-7bd61fae90ce@backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:18:55.705043 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:18:58.796735 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Differential backup transfer completed in 3.0924s {{(pid=134303) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Nov 20 21:18:58.797534 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Getting metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Nov 20 21:18:58.812207 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134303) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Nov 20 21:18:58.812690 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Getting metadata type 'volume-metadata' {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Nov 20 21:18:58.819901 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] No metadata type 'volume-metadata' available {{(pid=134303) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Nov 20 21:18:58.820281 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Getting metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Nov 20 21:18:58.828786 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134303) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Nov 20 21:18:58.829371 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Backing up metadata for volume 14992743-5006-407a-ba5a-7bd61fae90ce. {{(pid=134303) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Nov 20 21:18:58.863844 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Backup 'e2d330db-5233-464f-a223-c1b9124b8644' of volume 14992743-5006-407a-ba5a-7bd61fae90ce finished. {{(pid=134303) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Nov 20 21:18:58.865521 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d42e9d90>, 'connection_properties': {'name': 'volumes/volume-14992743-5006-407a-ba5a-7bd61fae90ce', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '14992743-5006-407a-ba5a-7bd61fae90ce', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d42cc670>}, 'force': True, 'ignore_errors': True}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:18:58.896136 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] <== disconnect_volume: return (29ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:18:58.956560 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-22720ad8-cf9a-4c4b-9e28-d6037846d6f7 tempest-TestVolumeBackupRestore-667839818 None] Create backup finished. backup: e2d330db-5233-464f-a223-c1b9124b8644.
Nov 20 21:19:00.497387 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Restore backup started, backup: e2d330db-5233-464f-a223-c1b9124b8644 volume: 534ec160-6718-4124-90f8-da2967763760.
Nov 20 21:19:00.510203 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.41', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:19:00.511607 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.521132 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:19:00.521420 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce17c8b6-67b0-4ff7-acdb-cb1e4450adba]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:3d404a852027\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:19:00.522764 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.528722 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f6c5f74c-9ecf-4912-aa20-d10b8685a30a]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:19:00.529476 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:19:00.530844 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6c5f74c-9ecf-4912-aa20-d10b8685a30a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:19:00.531987 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.538251 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[97863211-2e29-4746-9eed-7321a2a70343]: [Errno 2] No such file or directory: 'systool' {{(pid=170187) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]: Traceback (most recent call last):
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     ret = func(*f_args, **f_kwargs)
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     return func(*args, **kwargs)
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     return putils.execute(on_execute=on_execute,
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     obj = subprocess.Popen(cmd,
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     self._execute_child(args, executable, preexec_fn, close_fds,
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]:     raise child_exception_type(errno_num, err_msg, err_filename)
Nov 20 21:19:00.538873 ncdv-master-1 cinder-backup[134303]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Nov 20 21:19:00.540056 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[97863211-2e29-4746-9eed-7321a2a70343]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:19:00.540562 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.548994 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:19:00.549349 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[123e1cca-5c8b-4940-8128-b09246ce4dfa]: (4, ('/dev/sda1\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:19:00.550541 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.558283 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:19:00.558635 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[a444146a-c3ff-4f18-90f6-697fefe49480]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:19:00.559902 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.566110 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=170187) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:19:00.566620 ncdv-master-1 cinder-backup[134303]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0ea007e-185e-4a82-a0ef-c8a6a3f9f4cb]: (4, ('ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9\n', '')) {{(pid=170187) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Nov 20 21:19:00.567545 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Running cmd (subprocess): nvme version {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Nov 20 21:19:00.586399 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_concurrency.processutils [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=134303) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Nov 20 21:19:00.587429 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.utils [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] <== get_connector_properties: return (76ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.41', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:3d404a852027', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': 'ec8a1cee-95fc-dd43-884b-8fcae5e2d1a9', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:77af86bb-dfb9-4f30-b6b2-170584bd09e2'} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:19:01.280027 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connector [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Factory for rbd on None {{(pid=134303) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Nov 20 21:19:01.281481 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4c74070>, 'connection_properties': {'name': 'volumes/volume-534ec160-6718-4124-90f8-da2967763760', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '534ec160-6718-4124-90f8-da2967763760', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:19:01.282561 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.linuxrbd [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] opening connection to ceph cluster (timeout=-1). {{(pid=134303) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Nov 20 21:19:01.321617 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] <== connect_volume: return (39ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4e12940>} {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:19:01.329395 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:19:01.341863 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Starting restore from Ceph backup=e2d330db-5233-464f-a223-c1b9124b8644 to volume=volume-534ec160-6718-4124-90f8-da2967763760 {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Nov 20 21:19:01.401975 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Looking for snapshot of backup base 'volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:19:01.402539 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Found snapshot 'backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:19:01.411342 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Volume file is RBD. {{(pid=134303) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Nov 20 21:19:01.448112 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Attempting differential restore. {{(pid=134303) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Nov 20 21:19:01.448600 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Attempting incremental restore from base='volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' snap='backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436' {{(pid=134303) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Nov 20 21:19:01.449045 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Performing differential transfer from 'volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' to 'volume-534ec160-6718-4124-90f8-da2967763760' {{(pid=134303) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Nov 20 21:19:01.449448 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644@backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436 -' into... {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Nov 20 21:19:01.449814 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_6dlrot2n --pool volumes - volumes/volume-534ec160-6718-4124-90f8-da2967763760' {{(pid=134303) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Nov 20 21:19:04.987612 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Restore transfer completed in 3.5380s {{(pid=134303) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Nov 20 21:19:04.988657 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Nov 20 21:19:05.012139 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Restoring volume base metadata {{(pid=134303) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Nov 20 21:19:05.032148 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] No metadata of type 'volume-metadata' to restore {{(pid=134303) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Nov 20 21:19:05.032458 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.driver [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Restoring volume glance metadata {{(pid=134303) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Nov 20 21:19:05.171013 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Restore to volume 534ec160-6718-4124-90f8-da2967763760 finished successfully. {{(pid=134303) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Nov 20 21:19:05.173074 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7f67d4c74070>, 'connection_properties': {'name': 'volumes/volume-534ec160-6718-4124-90f8-da2967763760', 'hosts': ['10.222.0.41'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '534ec160-6718-4124-90f8-da2967763760', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7f67d4e12940>}, 'force': True, 'ignore_errors': False}" {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Nov 20 21:19:05.182558 ncdv-master-1 cinder-backup[134303]: DEBUG os_brick.initiator.connectors.rbd [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] <== disconnect_volume: return (8ms) None {{(pid=134303) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Nov 20 21:19:05.231994 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.manager [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Encryption key ID for volume 534ec160-6718-4124-90f8-da2967763760 already matches encryption key ID in backup e2d330db-5233-464f-a223-c1b9124b8644. {{(pid=134303) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Nov 20 21:19:05.261948 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-0a6d2edd-991f-443f-9ad6-9467958fdcae tempest-TestVolumeBackupRestore-667839818 None] Finished restoring backup e2d330db-5233-464f-a223-c1b9124b8644 to volume 534ec160-6718-4124-90f8-da2967763760.
Nov 20 21:19:54.599606 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:19:54.600221 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:20:11.669896 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Delete backup started, backup: e2d330db-5233-464f-a223-c1b9124b8644.
Nov 20 21:20:11.672400 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Delete started for backup=e2d330db-5233-464f-a223-c1b9124b8644 {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Nov 20 21:20:11.672893 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Trying diff format basename='volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' for backup base image of volume 14992743-5006-407a-ba5a-7bd61fae90ce. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Nov 20 21:20:11.805091 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Looking for snapshot of backup base 'volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Nov 20 21:20:11.806296 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Found snapshot 'backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436' {{(pid=134303) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Nov 20 21:20:11.806852 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Deleting backup snapshot='backup.e2d330db-5233-464f-a223-c1b9124b8644.snap.1637443135.1557436' {{(pid=134303) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Nov 20 21:20:12.730135 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Deleting backup base image='volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' of volume 14992743-5006-407a-ba5a-7bd61fae90ce.
Nov 20 21:20:13.443107 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Base backup image='volume-14992743-5006-407a-ba5a-7bd61fae90ce.backup.e2d330db-5233-464f-a223-c1b9124b8644' of volume 14992743-5006-407a-ba5a-7bd61fae90ce deleted. {{(pid=134303) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Nov 20 21:20:13.503919 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.backup.drivers.ceph [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Delete of backup 'e2d330db-5233-464f-a223-c1b9124b8644' for volume '14992743-5006-407a-ba5a-7bd61fae90ce' finished. {{(pid=134303) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Nov 20 21:20:13.510774 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_db.sqlalchemy.engines [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134303) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Nov 20 21:20:13.555635 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.quota [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Created reservations ['28f36b23-bdcd-434c-8e72-52f0d4087348', '397eecbf-5830-4416-bd02-762576a4bc48'] {{(pid=134303) reserve /opt/stack/cinder/cinder/quota.py:755}}
Nov 20 21:20:13.604473 ncdv-master-1 cinder-backup[134303]: INFO cinder.backup.manager [None req-322e4e6c-dbc6-404e-96ac-33db21aed13f tempest-TestVolumeBackupRestore-667839818 None] Delete backup finished, backup e2d330db-5233-464f-a223-c1b9124b8644 deleted.
Nov 20 21:20:54.596898 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:20:54.597594 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:21:54.593593 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:21:54.594335 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:22:54.589972 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:22:54.596570 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:23:54.587002 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:23:54.587623 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:24:54.592886 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:24:54.593956 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:25:54.589486 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:25:54.590260 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:26:54.595551 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:26:54.596140 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:27:54.600844 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:27:54.601409 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:28:54.605548 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:28:54.606210 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:29:54.611208 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:29:54.612360 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:30:54.607823 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:30:54.608429 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:31:54.613721 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:31:54.614326 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:32:54.610158 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:32:54.611139 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:33:54.606326 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:33:54.606976 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:34:54.610915 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:34:54.611598 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:35:54.608080 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:35:54.608730 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:36:54.613086 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:36:54.613721 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:37:54.617643 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:37:54.618352 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:38:54.624183 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:38:54.624867 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:39:54.628869 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:39:54.629555 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:40:54.626128 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:40:54.626824 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:41:54.631497 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:41:54.632186 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:42:54.636580 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:42:54.637284 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:43:54.642882 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:43:54.643592 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:44:54.638293 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:44:54.639009 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:45:54.643257 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:45:54.643979 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:46:54.640280 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:46:54.641000 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:47:54.646127 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:47:54.646884 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:48:54.641869 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:48:54.642596 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:49:54.647314 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:49:54.648019 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:50:54.652480 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:50:54.653155 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:51:54.658064 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:51:54.658811 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:52:54.655206 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:52:54.655910 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:53:54.660763 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:53:54.661467 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:54:54.656877 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:54:54.657597 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:55:54.653706 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:55:54.654454 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:56:54.651700 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:56:54.652393 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:57:54.656827 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:57:54.657521 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:58:54.653238 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:58:54.653891 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 21:59:54.657440 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 21:59:54.658150 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:00:54.655689 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:00:54.656366 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:01:54.661650 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:01:54.662389 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:02:54.667586 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:02:54.668254 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:03:54.673155 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:03:54.673854 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:04:54.678245 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:04:54.678906 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:05:54.683428 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:05:54.684099 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:06:54.688728 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:06:54.689352 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:07:54.694068 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:07:54.694726 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:08:54.699454 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:08:54.700104 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Nov 20 22:09:54.704937 ncdv-master-1 cinder-backup[134303]: DEBUG oslo_service.periodic_task [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134303) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Nov 20 22:09:54.705594 ncdv-master-1 cinder-backup[134303]: DEBUG cinder.manager [None req-63d09701-c1a5-4e47-8545-8b7a55067e95 None None] Notifying Schedulers of capabilities ... {{(pid=134303) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}