-- Logs begin at Sat 2022-01-29 20:01:13 UTC, end at Sat 2022-01-29 22:24:03 UTC. --
Jan 29 20:37:25.016710 ncdv-master-1 systemd[1]: Started Devstack devstack@c-bak.service.
Jan 29 20:37:28.380140 ncdv-master-1 cinder-backup[133750]: INFO cinder.cmd.backup [-] Backup running with 1 processes.
Jan 29 20:37:28.380928 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=133750) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}}
Jan 29 20:37:28.381348 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=133750) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}}
Jan 29 20:37:28.535601 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_db.sqlalchemy.engines [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133750) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 20:37:28.756038 ncdv-master-1 cinder-backup[133750]: INFO cinder.rpc [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Automatically selected cinder-backup objects version 1.39 as minimum service version.
Jan 29 20:37:28.759920 ncdv-master-1 cinder-backup[133750]: INFO cinder.rpc [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Automatically selected cinder-backup RPC version 2.2 as minimum service version.
Jan 29 20:37:28.764370 ncdv-master-1 cinder-backup[133750]: INFO cinder.rpc [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version.
Jan 29 20:37:28.767857 ncdv-master-1 cinder-backup[133750]: INFO cinder.rpc [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Automatically selected cinder-volume RPC version 3.16 as minimum service version.
Jan 29 20:37:28.772199 ncdv-master-1 cinder-backup[133750]: INFO cinder.rpc [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
Jan 29 20:37:28.776214 ncdv-master-1 cinder-backup[133750]: INFO cinder.rpc [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version.
Jan 29 20:37:28.825746 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_db.sqlalchemy.engines [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133750) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 20:37:28.846110 ncdv-master-1 cinder-backup[133750]: INFO oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Starting 1 workers
Jan 29 20:37:28.851717 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Started child 134197 {{(pid=133750) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Jan 29 20:37:28.852854 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Full set of CONF: {{(pid=133750) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}}
Jan 29 20:37:28.853232 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] ******************************************************************************** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}}
Jan 29 20:37:28.853552 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] Configuration options gathered from: {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}}
Jan 29 20:37:28.853908 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}}
Jan 29 20:37:28.854634 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}}
Jan 29 20:37:28.854943 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] ================================================================================ {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}}
Jan 29 20:37:28.855828 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] allow_availability_zone_fallback = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.856260 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] allow_compression_on_image_upload = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.856856 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] allowed_direct_url_schemes     = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.857313 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] api_rate_limit                 = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.857783 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] auth_strategy                  = keystone {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.858170 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_availability_zone      = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.858566 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_api_class               = cinder.backup.api.API {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.859067 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_chunk_size         = 134217728 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.859509 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_conf               = /etc/ceph/ceph.conf {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.859882 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_image_journals     = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.860287 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_pool               = backups {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.860677 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_stripe_count       = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_stripe_unit        = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_ceph_user               = cinder-bak {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_driver                  = cinder.backup.drivers.ceph.CephBackupDriver {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_driver_init_check_interval = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_driver_stats_polling_interval = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_manager                 = cinder.backup.manager.BackupManager {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863879 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_max_operations          = 15 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.863979 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_metadata_version        = 2 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.864398 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_name_template           = backup-%s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.864663 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_native_threads_pool_size = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.865035 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_object_number_per_notification = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.865528 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_service_inithost_offload = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.865892 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_timer_interval          = 120 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.866273 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_use_same_host           = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.866643 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_use_temp_snapshot       = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.867044 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backup_workers                 = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.867434 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] chap_password                  = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.867806 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] chap_username                  =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.868191 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.868629 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_internal_tenant_project_id = 190d9901ea4943389237dfe2aab638ae {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.869059 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_internal_tenant_user_id = b545067a01a24e70a9e19abc1cc76392 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.869439 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] compression_format             = gzip {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.869885 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] config_dir                     = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.870194 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] config_file                    = ['/etc/cinder/cinder.conf'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.870687 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] config_source                  = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.871092 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] consistencygroup_api_class     = cinder.consistencygroup.api.API {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.872216 ncdv-master-1 cinder-backup[134197]: INFO cinder.service [-] Starting cinder-backup node (version 19.1.0)
Jan 29 20:37:28.872759 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] control_exchange               = openstack {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.873131 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] debug                          = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.873501 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] default_availability_zone      = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.873849 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] default_group_type             = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.874469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'swiftclient=WARN', 'botocore=WARN'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.874999 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] default_volume_type            = ceph {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.875402 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] driver_client_cert             = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.875885 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] driver_client_cert_key         = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.876239 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] driver_data_namespace          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.876595 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] driver_ssl_cert_path           = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.876936 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] driver_ssl_cert_verify         = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.877355 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] driver_use_ssl                 = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.877659 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] enable_new_services            = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880212 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] enable_unsupported_driver      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880212 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] enabled_backends               = ['ceph'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880212 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] enforce_multipath_for_image_xfer = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880212 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] filter_function                = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880212 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_api_insecure            = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880212 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_api_servers             = ['http://10.222.0.20/image'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880635 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_api_ssl_compression     = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.880635 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_ca_certificates_file    = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.881140 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_catalog_info            = image:glance:publicURL {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.881325 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_certfile                = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.881940 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.882308 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_keyfile                 = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.882674 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_num_retries             = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.883013 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] glance_request_timeout         = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.883427 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] goodness_function              = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.883808 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] graceful_shutdown_timeout      = 5 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.884178 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] group_api_class                = cinder.group.api.API {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.884550 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] host                           = ncdv-master-1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.885139 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iet_conf                       = /etc/iet/ietd.conf {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.885817 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_compress_on_upload       = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.886268 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_conversion_address_space_limit = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.886634 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_conversion_cpu_limit     = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.887471 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_conversion_dir           = /opt/stack/data/cinder/conversion {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.887870 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_upload_use_cinder_backend = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.888298 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_upload_use_internal_tenant = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.888773 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_volume_cache_enabled     = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.889153 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_volume_cache_max_count   = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.889564 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] image_volume_cache_max_size_gb = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.890420 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] instance_format                = [instance: %(uuid)s]  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.890881 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.891420 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iscsi_iotype                   = fileio {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.891955 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iscsi_secondary_ip_addresses   = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.892462 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iscsi_target_flags             =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.892851 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iscsi_write_cache              = on {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.893486 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iser_helper                    = tgtadm {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.894175 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iser_ip_address                = 10.222.0.20 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.894654 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iser_port                      = 3260 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.895021 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] iser_target_prefix             = iqn.2010-10.org.openstack: {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.895420 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_config_append              = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.896083 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.896469 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_dir                        = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.896965 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_file                       = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.900854 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_options                    = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.901277 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_rotate_interval            = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.901699 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_rotate_interval_type       = days {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.902225 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] log_rotation_type              = none {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.902538 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.903344 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.903732 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.904092 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.904383 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.904752 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] max_age                        = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.905150 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] max_logfile_count              = 30 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.905501 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] max_logfile_size_mb            = 200 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.906015 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] max_over_subscription_ratio    = 20.0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.906421 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] message_reap_interval          = 86400 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.906792 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] message_ttl                    = 2592000 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.907173 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] monkey_patch                   = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.907531 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] monkey_patch_modules           = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.907839 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] my_ip                          = 10.222.0.20 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.908183 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] no_snapshot_gb_quota           = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.908595 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] num_iser_scan_tries            = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.909007 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] num_shell_tries                = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.909407 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] num_volume_device_scan_tries   = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.909784 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] nvmet_ns_id                    = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.910133 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] nvmet_port_id                  = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.910525 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_max_limit                = 1000 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.910873 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_volume_ext_list          = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.911280 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.911643 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_volume_listen            = 0.0.0.0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.912120 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_volume_listen_port       = 8776 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.912474 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_volume_use_ssl           = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.912832 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] osapi_volume_workers           = 2 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.913206 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] per_volume_size_limit          = -1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.913550 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] periodic_fuzzy_delay           = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.913912 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] periodic_interval              = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.914249 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] public_endpoint                = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.914610 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] publish_errors                 = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.914973 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_backup_gigabytes         = 1000 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.915359 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_backups                  = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.915738 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_consistencygroups        = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.916221 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_driver                   = cinder.quota.DbQuotaDriver {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.916588 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_gigabytes                = 1000 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.916932 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_groups                   = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.919898 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_snapshots                = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.919898 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] quota_volumes                  = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.919898 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] rate_limit_burst               = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.919898 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] rate_limit_except_level        = CRITICAL {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.920562 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] rate_limit_interval            = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.920562 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] replication_device             = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.921960 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] report_discard_supported       = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.921960 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] report_interval                = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.922297 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] reservation_clean_interval     = 86400 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.922729 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] reservation_expire             = 86400 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.922919 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] reserved_percentage            = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.925322 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] resource_query_filters_file    = /etc/cinder/resource_filters.json {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.925322 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] restore_discard_excess_bytes   = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.925322 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] rootwrap_config                = /etc/cinder/rootwrap.conf {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.925322 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] rpc_response_timeout           = 120 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.925322 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] run_external_periodic_tasks    = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.925322 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] scheduler_manager              = cinder.scheduler.manager.SchedulerManager {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.926206 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] scst_target_driver             = iscsi {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.926206 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] scst_target_iqn_name           = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.926206 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_down_time              = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.926626 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] snapshot_name_template         = snapshot-%s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.927380 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] split_loggers                  = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.927380 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] state_path                     = /opt/stack/data/cinder {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.927695 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] storage_availability_zone      = nova {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.932600 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] storage_protocol               = iscsi {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.932600 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] syslog_log_facility            = LOG_USER {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.932600 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] target_helper                  = lioadm {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.932600 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] target_ip_address              = 10.222.0.20 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.932600 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] target_port                    = 3260 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.932600 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] target_prefix                  = iqn.2010-10.org.openstack: {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] target_protocol                = iscsi {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] trace_flags                    = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] transfer_api_class             = cinder.transfer.api.API {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] transport_url                  = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] until_refresh                  = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_chap_auth                  = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.933573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_default_quota_class        = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_eventlog                   = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_journal                    = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_json                       = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_multipath_for_image_xfer   = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_stderr                     = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935749 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] use_syslog                     = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.935749 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] verify_glance_signatures       = enabled {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.936019 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_api_class               = cinder.volume.api.API {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.936396 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_backend_name            = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.936920 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_clear                   = zero {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.937311 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_clear_ionice            = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.937775 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_clear_size              = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.938457 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_copy_blkio_cgroup_name  = cinder-volume-copy {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.938826 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.lockutils [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] Lock "cleanup_incomplete_backups_133750" acquired by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: waited 0.001s {{(pid=134197) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}}
Jan 29 20:37:28.939174 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] Cleaning up incomplete backup operations.
Jan 29 20:37:28.939536 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_copy_bps_limit          = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.940071 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_dd_blocksize            = 1M {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.940512 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_manager                 = cinder.volume.manager.VolumeManager {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.940891 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_name_template           = volume-%s {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.941291 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volume_usage_audit_period      = month {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.941777 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] volumes_dir                    = /opt/stack/data/cinder/volumes {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.942130 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] watch_log_file                 = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jan 29 20:37:28.942811 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.943216 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.943614 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.enforce_new_defaults = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.943994 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.enforce_scope      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.944362 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.policy_default_rule = default {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.945115 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.945521 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.policy_file        = policy.yaml {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.945915 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.946299 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.946807 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.947155 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.947513 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.947878 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_concurrency.disable_process_locking = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.948338 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_concurrency.lock_path     = /opt/stack/data/cinder {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.948732 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.admin_password = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.949139 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.949501 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.admin_token = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.950885 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.admin_user  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.951421 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_admin_prefix =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.951937 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_host   = 127.0.0.1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.952456 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_port   = 35357 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.953201 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_protocol = https {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.953697 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_section = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.954154 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_type   = password {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.954660 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_uri    = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.955189 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.auth_version = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.955720 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.cache       = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.956232 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.cafile      = /opt/stack/data/ca-bundle.pem {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.957134 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.certfile    = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.957398 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.delay_auth_decision = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.957782 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.958615 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.http_connect_timeout = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.958870 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.960519 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.identity_uri = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.960519 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.include_service_catalog = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.960519 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.insecure    = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.961369 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.interface   = public {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.961369 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.keyfile     = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.961369 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.962066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.962066 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.962552 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.964616 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.964616 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.964616 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.964616 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.964616 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.965932 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.region_name = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.965932 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.965932 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.service_token_roles_required = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.965932 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.service_type = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.966606 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.token_cache_time = 300 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.966606 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.967441 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.backend                  = dogpile.cache.null {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.967613 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.backend_argument         = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.969116 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.config_prefix            = cache.oslo {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.969116 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.debug_cache_backend      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.969116 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.enabled                  = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.969116 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.expiration_time          = 600 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.970000 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_dead_retry      = 300 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.970527 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.971024 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.971842 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_pool_maxsize    = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.972336 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.973046 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.973671 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.974442 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.proxies                  = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.975123 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.tls_allowed_ciphers      = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.975842 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.tls_cafile               = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.976400 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.tls_certfile             = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.977002 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.tls_enabled              = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.978006 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cache.tls_keyfile              = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.978678 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] coordination.backend_url       = etcd3+http://10.222.0.20:2379 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.979450 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.backend               = sqlalchemy {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.980069 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.connection            = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.980653 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.connection_debug      = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.981444 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.connection_parameters =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.981977 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.connection_recycle_time = 3600 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.982530 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.connection_trace      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.983224 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.db_inc_retry_interval = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.983743 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.db_max_retries        = 20 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.984275 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.db_max_retry_interval = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.984814 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.db_retry_interval     = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.985371 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.max_overflow          = 50 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.985927 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.max_pool_size         = 5 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.986483 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.max_retries           = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.987016 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.mysql_enable_ndb      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.987573 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.988243 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.pool_timeout          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.988797 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.retry_interval        = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.989346 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.slave_connection      = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.989903 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.sqlite_synchronous    = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.990448 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] database.use_db_reconnect      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.991037 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.991664 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.992209 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.992749 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.993395 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.993977 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.connection_string     = messaging:// {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.994459 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.enabled               = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.995019 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.es_doc_type           = notification {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.995549 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.es_scroll_size        = 10000 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.996091 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.es_scroll_time        = 2m {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.997163 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.filter_error_trace    = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.997744 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.998393 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.sentinel_service_name = mymaster {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.998934 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.socket_timeout        = 0.1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:28.999496 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] profiler.trace_sqlalchemy      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.000103 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] privsep_osbrick.capabilities   = [21] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.000645 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] privsep_osbrick.group          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.001234 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] privsep_osbrick.helper_command = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.001802 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] privsep_osbrick.logger_name    = oslo_privsep.daemon {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.002342 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.002895 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] privsep_osbrick.user           = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.003564 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_sys_admin.capabilities  = [0, 1, 2, 3, 12, 21] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.004092 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_sys_admin.group         = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.004631 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_sys_admin.helper_command = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.005202 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_sys_admin.logger_name   = oslo_privsep.daemon {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.005738 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.006278 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] cinder_sys_admin.user          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.006842 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.auth_section      = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.007409 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.auth_type         = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.007928 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.cafile            = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.008701 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.certfile          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.009317 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.collect_timing    = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.009846 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.insecure          = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.010387 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.keyfile           = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.011120 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.send_service_user_token = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.011661 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.split_loggers     = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.012174 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] service_user.timeout           = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.013437 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.backend_availability_zone = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.013966 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.chap_password = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.014545 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.chap_username =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.015087 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.015622 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.016141 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.016694 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.driver_client_cert = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.017337 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.driver_client_cert_key = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.017923 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.driver_data_namespace = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.018579 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.019128 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.019662 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.driver_use_ssl = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.020269 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.enable_deferred_deletion = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.020795 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.enable_unsupported_driver = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.021375 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.021896 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.filter_function = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.022434 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.goodness_function = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.023010 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iet_conf      = /etc/iet/ietd.conf {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.023660 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.024191 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.024719 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.025435 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.025966 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.026525 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iscsi_iotype  = fileio {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.027076 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.027609 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iscsi_target_flags =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.028516 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iscsi_write_cache = on {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.029093 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iser_helper   = tgtadm {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.029845 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iser_ip_address = 10.222.0.20 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.030394 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iser_port     = 3260 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.030941 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.031574 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.032119 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.033822 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.num_shell_tries = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.034473 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.035021 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.nvmet_ns_id   = 10 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.035592 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.nvmet_port_id = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.036156 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.036701 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rados_connection_interval = 5 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.037275 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rados_connection_retries = 3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.037822 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_ceph_conf =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.038425 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.039078 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.039608 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.040152 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.040701 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_pool      = rbd {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.041262 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_secret_uuid = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.041799 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.042343 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.rbd_user      = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.042896 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.043424 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.replication_device = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.044624 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.report_discard_supported = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.045190 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.045723 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.reserved_percentage = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.046265 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.scst_target_driver = iscsi {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.047060 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.scst_target_iqn_name = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.047713 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.storage_protocol = iscsi {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.048264 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.target_helper = tgtadm {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.049074 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.target_ip_address = 10.222.0.20 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.049609 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.target_port   = 3260 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.050166 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.050713 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.target_protocol = iscsi {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.051246 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.trace_flags   = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.051772 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.unique_fqdn_network = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.052298 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.use_chap_auth = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.052860 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.053420 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_backend_name = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.053965 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_clear  = zero {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.054587 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_clear_ionice = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.055126 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_clear_size = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.055665 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.056211 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.056748 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.057450 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] backend_defaults.volumes_dir   = /opt/stack/data/cinder/volumes {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.058011 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_reports.file_event_handler = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.058575 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.059819 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_reports.log_dir           = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.060483 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.061116 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.061705 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.062266 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.062862 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.063420 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.063988 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.064644 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.065260 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.065810 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.066407 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.066970 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.067569 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.068125 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.068691 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.069420 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.069975 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.070558 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.071118 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.071700 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.072273 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.072833 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.ssl      = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.073439 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.074024 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.074750 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.075349 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.076045 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_notifications.driver = [] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.076621 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_notifications.retry = -1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.077382 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.077952 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] oslo_messaging_notifications.transport_url = **** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.078588 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] key_manager.backend            = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.079356 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.079902 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.barbican_api_version  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.080488 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.barbican_endpoint     = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.081109 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.barbican_endpoint_type = public {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.081754 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.barbican_region_name  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.082319 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.cafile                = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.082873 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.certfile              = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.083464 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.collect_timing        = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.084015 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.insecure              = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.084564 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.keyfile               = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.085348 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.number_of_retries     = 60 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.085902 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.retry_delay           = 1 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.086453 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.send_service_user_token = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.087032 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.split_loggers         = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.087585 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.timeout               = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.088134 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.verify_ssl            = True {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.088691 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican.verify_ssl_path       = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.089397 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.auth_section = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.090176 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.auth_type = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.090750 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.cafile   = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.091302 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.certfile = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.091926 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.collect_timing = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.092485 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.insecure = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.093075 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.keyfile  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.093642 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.split_loggers = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.094203 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] barbican_service_user.timeout  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.094764 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.approle_role_id          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.095412 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.approle_secret_id        = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.095955 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.cafile                   = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.096505 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.certfile                 = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.097110 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.collect_timing           = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.097665 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.insecure                 = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.098222 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.keyfile                  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.098946 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.kv_mountpoint            = secret {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.099611 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.kv_version               = 2 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.100161 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.namespace                = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.100876 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.root_token_id            = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.101478 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.split_loggers            = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.102051 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.ssl_ca_crt_file          = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.102592 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.timeout                  = None {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.103157 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.use_ssl                  = False {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.103732 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jan 29 20:37:29.104216 ncdv-master-1 cinder-backup[133750]: DEBUG oslo_service.service [None req-9490b329-fb8a-4ae6-82b3-7fa5a86f4333 None None] ******************************************************************************** {{(pid=133750) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}}
Jan 29 20:37:29.175839 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 20:37:29.208885 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.lockutils [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] Lock "cleanup_incomplete_backups_133750" "released" by "cinder.backup.manager.BackupManager._cleanup_incomplete_backup_operations" :: held 0.270s {{(pid=134197) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}}
Jan 29 20:37:29.221802 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 20:37:29.228080 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:37:29.247781 ncdv-master-1 cinder-backup[134197]: INFO cinder.keymgr.migration [None req-67e792dd-b8b4-4810-85b6-ce4cf3b2ebeb None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Jan 29 20:37:29.270682 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.service [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] Creating RPC server for service cinder-backup {{(pid=134197) start /opt/stack/cinder/cinder/service.py:231}}
Jan 29 20:37:29.281550 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.service [None req-30736e9c-6220-4558-bd33-607bfde49115 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=134197) start /opt/stack/cinder/cinder/service.py:237}}
Jan 29 20:38:31.335262 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:38:31.335895 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:39:31.331338 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:39:31.332029 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:41:31.336917 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:41:31.337552 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:42:31.341236 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:42:31.341836 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:43:31.336812 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:43:31.337447 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:44:31.341506 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:44:31.342142 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:45:31.346364 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:45:31.346991 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:46:31.351425 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:46:31.352121 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:47:31.348583 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:47:31.349148 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:48:31.344905 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:48:31.345568 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:49:31.355427 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:49:31.355427 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:50:31.352319 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:50:31.353049 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:51:31.353307 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:51:31.354320 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:52:31.354042 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:52:31.355416 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:53:31.350528 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:53:31.351717 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:54:31.347256 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:54:31.347917 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:55:31.353984 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:55:31.354660 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:56:31.351687 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:56:31.352347 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:57:31.357189 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:57:31.357189 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:58:31.354304 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:58:31.355020 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 20:59:31.351416 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 20:59:31.352107 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:00:31.351964 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:00:31.352691 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:01:31.351387 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:01:31.353115 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:02:31.347772 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:02:31.348414 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:03:31.347382 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:03:31.348502 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:04:31.347533 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:04:31.348846 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:05:01.748910 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Create backup started, backup: 9613b9c4-3653-4464-a9fc-7901e03542b8 volume: 030e378f-c9fa-4e82-9f25-1c298c1a9cb5.
Jan 29 21:05:01.765293 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:05:01.768094 ncdv-master-1 cinder-backup[134197]: INFO oslo.privsep.daemon [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp8wlu_i1p/privsep.sock']
Jan 29 21:05:01.924470 ncdv-master-1 sudo[160415]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmp8wlu_i1p/privsep.sock
Jan 29 21:05:01.938457 ncdv-master-1 sudo[160415]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jan 29 21:05:08.079222 ncdv-master-1 sudo[160415]: pam_unix(sudo:session): session closed for user root
Jan 29 21:05:08.090888 ncdv-master-1 cinder-backup[134197]: INFO oslo.privsep.daemon [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Spawned new privsep daemon via rootwrap
Jan 29 21:05:08.091804 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Accepted privsep connection to /tmp/tmp8wlu_i1p/privsep.sock {{(pid=134197) __init__ /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:363}}
Jan 29 21:05:08.094315 ncdv-master-1 cinder-backup[134197]: INFO oslo.privsep.daemon [-] privsep daemon starting
Jan 29 21:05:08.094802 ncdv-master-1 cinder-backup[134197]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Jan 29 21:05:08.095697 ncdv-master-1 cinder-backup[134197]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Jan 29 21:05:08.096529 ncdv-master-1 cinder-backup[134197]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 160493
Jan 29 21:05:08.099317 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe1e93e8-ffb1-49c7-9838-2c3edd976ea9]: (2,) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:08.757805 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:08.793444 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.035s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:08.794297 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[8472bf99-2c07-451f-ba08-63d201da007f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:08.799238 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:08.822433 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ec7b258b-8177-4aea-9a05-69e363b83df3]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:05:08.840674 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:05:08.844202 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec7b258b-8177-4aea-9a05-69e363b83df3]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:08.844202 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:08.869441 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a9f6325f-bc49-4dd6-bc17-3c786a9eae36]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:05:08.870037 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:05:08.872575 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9f6325f-bc49-4dd6-bc17-3c786a9eae36]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:08.875498 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:08.933046 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.057s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:08.934829 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[a01c1203-7d5e-40a5-9caf-bdc5212dedc8]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:08.936540 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:09.022644 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.086s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:09.023288 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[929c4265-ca9c-4af1-9c91-fb921f8fabee]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:09.028193 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:09.060421 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.034s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:09.061505 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[597f96d4-e429-4954-b023-1f016dc72c7e]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:09.062959 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:09.104711 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] CMD "nvme version" returned: 0 in 0.042s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:09.112789 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] <== get_connector_properties: return (7345ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:05:09.466926 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-be73d8a1-2210-4b9c-9c60-e9e3779e5d38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:05:10.343515 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:05:10.346930 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6e82bedc0>, 'connection_properties': {'name': 'volumes/volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '030e378f-c9fa-4e82-9f25-1c298c1a9cb5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:05:10.348778 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:05:10.497512 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] <== connect_volume: return (149ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca51e50>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:05:10.530826 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Starting backup of volume='030e378f-c9fa-4e82-9f25-1c298c1a9cb5'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:05:10.533723 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Volume file is RBD: attempting optimized backup {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 29 21:05:10.610391 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Creating base image 'volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' {{(pid=134197) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 29 21:05:10.611130 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:05:12.807447 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Using --from-snap 'None' for incremental backup of volume 030e378f-c9fa-4e82-9f25-1c298c1a9cb5. {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 29 21:05:12.808236 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Creating backup snapshot='backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706' {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 29 21:05:13.786307 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Performing differential transfer from 'volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5' to 'volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:05:13.787456 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_4o50ilds --pool volumes volumes/volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5@backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:05:13.787776 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:05:23.258000 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Differential backup transfer completed in 9.4712s {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 29 21:05:23.259108 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:05:23.284648 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:05:23.285591 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:05:23.305789 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:05:23.306190 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:05:23.325786 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Jan 29 21:05:23.326419 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Backing up metadata for volume 030e378f-c9fa-4e82-9f25-1c298c1a9cb5. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:05:23.426778 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Backup '9613b9c4-3653-4464-a9fc-7901e03542b8' of volume 030e378f-c9fa-4e82-9f25-1c298c1a9cb5 finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:05:23.428705 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6e82bedc0>, 'connection_properties': {'name': 'volumes/volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '030e378f-c9fa-4e82-9f25-1c298c1a9cb5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca51e50>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:05:23.465137 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] <== disconnect_volume: return (35ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:05:23.597132 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-055278ad-444d-4e00-a70e-5e06753a2261 tempest-TestVolumeBackupRestore-1484692939 None] Create backup finished. backup: 9613b9c4-3653-4464-a9fc-7901e03542b8.
Jan 29 21:05:25.361853 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Restore backup started, backup: 9613b9c4-3653-4464-a9fc-7901e03542b8 volume: 7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0.
Jan 29 21:05:25.393527 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:05:25.399434 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.417780 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.022s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:25.418119 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[24f5ba11-f9b8-4495-a179-7d2fea6d1bb1]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:25.419953 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.434058 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[68ebd0b4-0a3d-405f-8d63-60f1c7014f14]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:05:25.434387 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:05:25.436550 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[68ebd0b4-0a3d-405f-8d63-60f1c7014f14]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:25.441017 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.454260 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8b49b50f-c2e0-48e5-aecd-81751889e0a2]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:05:25.456219 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:05:25.458685 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b49b50f-c2e0-48e5-aecd-81751889e0a2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:25.460073 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.478554 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.018s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:25.479043 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[29f528be-a5cc-4911-9cdc-5bf861211a68]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:25.480951 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.503942 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.022s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:25.504363 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cd31988-0404-4867-9b3c-ae1b9ca9e29a]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:25.507400 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.526818 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.019s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:25.527133 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[8347c687-e276-4718-bf7c-a8b63a8b0e23]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:05:25.528042 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:05:25.555850 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] CMD "nvme version" returned: 0 in 0.027s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:05:25.557794 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] <== get_connector_properties: return (163ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:05:26.591734 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:05:26.593287 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc9e51c0>, 'connection_properties': {'name': 'volumes/volume-7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:05:26.594644 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:05:26.666397 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] <== connect_volume: return (72ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc9e5c10>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:05:26.673944 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:05:26.688666 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Starting restore from Ceph backup=9613b9c4-3653-4464-a9fc-7901e03542b8 to volume=volume-7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0 {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Jan 29 21:05:26.821096 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Looking for snapshot of backup base 'volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:05:26.821805 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Found snapshot 'backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:05:26.829058 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Volume file is RBD. {{(pid=134197) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Jan 29 21:05:26.903699 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Attempting differential restore. {{(pid=134197) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Jan 29 21:05:26.905376 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Attempting incremental restore from base='volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' snap='backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706' {{(pid=134197) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Jan 29 21:05:26.905376 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Performing differential transfer from 'volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' to 'volume-7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:05:26.905376 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8@backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:05:26.905376 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_vx4bh90p --pool volumes - volumes/volume-7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:05:31.344692 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:05:31.346442 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:05:33.359930 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Restore transfer completed in 6.4552s {{(pid=134197) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Jan 29 21:05:33.362716 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Jan 29 21:05:33.432371 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Restoring volume base metadata {{(pid=134197) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Jan 29 21:05:33.468106 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] No metadata of type 'volume-metadata' to restore {{(pid=134197) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 29 21:05:33.468594 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Restoring volume glance metadata {{(pid=134197) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Jan 29 21:05:33.782475 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Restore to volume 7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0 finished successfully. {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Jan 29 21:05:33.784659 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc9e51c0>, 'connection_properties': {'name': 'volumes/volume-7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc9e5c10>}, 'force': True, ': False}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:05:33.805671 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] <== disconnect_volume: return (19ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:05:33.939621 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.manager [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Encryption key ID for volume 7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0 already matches encryption key ID in backup 9613b9c4-3653-4464-a9fc-7901e03542b8. {{(pid=134197) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Jan 29 21:05:33.991731 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-8ce1509c-094b-44db-85a6-dc97dd78ae88 tempest-TestVolumeBackupRestore-1484692939 None] Finished restoring backup 9613b9c4-3653-4464-a9fc-7901e03542b8 to volume 7e8cfa75-20c6-4bb8-ba8f-f16b6befc7a0.
Jan 29 21:06:31.341491 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:06:31.342870 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:07:05.926543 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Delete backup started, backup: 9613b9c4-3653-4464-a9fc-7901e03542b8.
Jan 29 21:07:05.930358 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Delete started for backup=9613b9c4-3653-4464-a9fc-7901e03542b8 {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:07:05.931126 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Trying diff format basename='volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' for backup base image of volume 030e378f-c9fa-4e82-9f25-1c298c1a9cb5. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:07:06.038625 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Looking for snapshot of backup base 'volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:07:06.039049 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Found snapshot 'backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:07:06.039423 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Deleting backup snapshot='backup.9613b9c4-3653-4464-a9fc-7901e03542b8.snap.1643490312.8074706' {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 29 21:07:06.603228 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Deleting backup base image='volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' of volume 030e378f-c9fa-4e82-9f25-1c298c1a9cb5.
Jan 29 21:07:07.579638 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Base backup image='volume-030e378f-c9fa-4e82-9f25-1c298c1a9cb5.backup.9613b9c4-3653-4464-a9fc-7901e03542b8' of volume 030e378f-c9fa-4e82-9f25-1c298c1a9cb5 deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:07:07.633491 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Delete of backup '9613b9c4-3653-4464-a9fc-7901e03542b8' for volume '030e378f-c9fa-4e82-9f25-1c298c1a9cb5' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:07:07.647731 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:07:07.799701 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Created reservations ['450839c1-73ce-4080-84a3-a87f93edb64f', '460c81b8-4e55-4551-8d74-1fea245004df'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:07:07.874022 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-66eef875-ff30-4a63-89b4-fc3f7f34ccff tempest-TestVolumeBackupRestore-1484692939 None] Delete backup finished, backup 9613b9c4-3653-4464-a9fc-7901e03542b8 deleted.
Jan 29 21:07:31.340331 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:07:31.340945 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:08:31.336640 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:08:31.337293 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:09:31.337009 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:09:31.337602 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:10:31.336302 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:10:31.337049 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:11:31.336712 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:11:31.337429 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:12:31.341667 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:12:31.342245 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:13:03.260234 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Create backup started, backup: 3c9a5b02-33ee-49fe-bad2-3e280696255a volume: f0830cc4-21e5-4427-b434-f0f0f2ae32e7.
Jan 29 21:13:03.278911 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:03.282807 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.308516 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.024s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:03.309629 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[f738466d-6de2-4f14-bb03-4d7c82e118fa]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:03.313052 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.334935 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6bac7dd3-61b9-45be-b38d-876d6c41845d]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:03.336178 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:03.337656 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bac7dd3-61b9-45be-b38d-876d6c41845d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:03.338742 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.347264 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[514e6d1d-4f8c-4563-9da7-c76fcee4177b]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:03.347923 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:03.349898 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[514e6d1d-4f8c-4563-9da7-c76fcee4177b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:03.353081 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.387297 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.036s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:03.387635 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[7993c0a2-c0d8-4b15-bed5-c3c707e76191]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:03.389098 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.430011 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.041s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:03.430505 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7e56ba7-961b-476d-b24a-46bfd8667da6]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:03.433894 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.478788 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.044s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:03.479087 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7b4d51d-2180-4dbe-8e22-ea97fdc3e85b]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:03.479851 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:03.551317 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] CMD "nvme version" returned: 0 in 0.071s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:03.552466 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] <== get_connector_properties: return (272ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:04.906671 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:13:04.908190 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dcbb85e0>, 'connection_properties': {'name': 'volumes/volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f0830cc4-21e5-4427-b434-f0f0f2ae32e7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:04.910125 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:13:05.057581 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] <== connect_volume: return (148ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca69280>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:05.065045 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:05.105131 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Starting backup of volume='f0830cc4-21e5-4427-b434-f0f0f2ae32e7'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:13:05.105994 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Volume file is RBD: attempting optimized backup {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 29 21:13:05.162857 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Creating base image 'volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' {{(pid=134197) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 29 21:13:05.163419 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:13:05.234399 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Using --from-snap 'None' for incremental backup of volume f0830cc4-21e5-4427-b434-f0f0f2ae32e7. {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 29 21:13:05.235476 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Creating backup snapshot='backup.3c9a5b02-33ee-49fe-bad2-3e280696255a.snap.1643490785.2344165' {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 29 21:13:06.223821 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Performing differential transfer from 'volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7' to 'volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:13:06.225523 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_4l3qhho0 --pool volumes volumes/volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7@backup.3c9a5b02-33ee-49fe-bad2-3e280696255a.snap.1643490785.2344165 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:13:06.225910 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:13:07.303393 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Differential backup transfer completed in 1.0784s {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 29 21:13:07.304036 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:13:07.329904 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:13:07.330315 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:13:07.345831 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:13:07.346317 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:13:07.356238 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] No metadata type 'volume-glance-metadata' available {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 29 21:13:07.359213 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Backing up metadata for volume f0830cc4-21e5-4427-b434-f0f0f2ae32e7. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:13:07.436222 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Backup '3c9a5b02-33ee-49fe-bad2-3e280696255a' of volume f0830cc4-21e5-4427-b434-f0f0f2ae32e7 finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:13:07.441634 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dcbb85e0>, 'connection_properties': {'name': 'volumes/volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'f0830cc4-21e5-4427-b434-f0f0f2ae32e7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca69280>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:07.462612 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] <== disconnect_volume: return (20ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:07.578736 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-d3c233b5-7063-45a7-ac82-ccfb8e86e241 tempest-VolumesBackupsTest-2013611801 None] Create backup finished. backup: 3c9a5b02-33ee-49fe-bad2-3e280696255a.
Jan 29 21:13:08.620768 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Delete backup started, backup: 3c9a5b02-33ee-49fe-bad2-3e280696255a.
Jan 29 21:13:08.624010 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Delete started for backup=3c9a5b02-33ee-49fe-bad2-3e280696255a {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:13:08.624697 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Trying diff format basename='volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' for backup base image of volume f0830cc4-21e5-4427-b434-f0f0f2ae32e7. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:13:08.754242 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Looking for snapshot of backup base 'volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:13:08.754914 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Found snapshot 'backup.3c9a5b02-33ee-49fe-bad2-3e280696255a.snap.1643490785.2344165' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:13:08.755021 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Deleting backup snapshot='backup.3c9a5b02-33ee-49fe-bad2-3e280696255a.snap.1643490785.2344165' {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 29 21:13:09.336952 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Deleting backup base image='volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' of volume f0830cc4-21e5-4427-b434-f0f0f2ae32e7.
Jan 29 21:13:09.523305 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-be73d8a1-2210-4b9c-9c60-e9e3779e5d38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:09.581201 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Base backup image='volume-f0830cc4-21e5-4427-b434-f0f0f2ae32e7.backup.3c9a5b02-33ee-49fe-bad2-3e280696255a' of volume f0830cc4-21e5-4427-b434-f0f0f2ae32e7 deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:13:09.637465 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Delete of backup '3c9a5b02-33ee-49fe-bad2-3e280696255a' for volume 'f0830cc4-21e5-4427-b434-f0f0f2ae32e7' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:13:09.669381 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Created reservations ['ab925c52-c3e2-4a14-8b0f-f2a183b12aa3', '05586e07-31e9-4e89-9563-0cf31d554029'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:13:09.724748 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-09086e61-9037-4e71-9b05-e51b742b6816 tempest-VolumesBackupsTest-2013611801 None] Delete backup finished, backup 3c9a5b02-33ee-49fe-bad2-3e280696255a deleted.
Jan 29 21:13:19.360211 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Create backup started, backup: 318bf7e5-1cf6-4993-9ab9-636adc9dda50 volume: 9723b5a7-1252-49d7-b3f1-ae5426460e67.
Jan 29 21:13:19.368212 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:19.369701 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.377086 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:19.377629 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[c40e6418-8269-4404-9c7b-30cfd545f78d]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:19.378938 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.384865 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[01022cce-f4a7-41b6-9378-eaec8b3bc671]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:19.385628 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:19.386916 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[01022cce-f4a7-41b6-9378-eaec8b3bc671]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:19.387423 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.393495 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0026b6d6-0487-4bcd-92bf-5f67764f7846]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:19.394804 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:19.396676 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[0026b6d6-0487-4bcd-92bf-5f67764f7846]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:19.397195 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.405909 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.009s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:19.406352 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d938572-c782-4f2e-b5bf-e5a791498f5d]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:19.407562 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.416580 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:19.417143 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e21d96e-8a59-404b-adc8-35329009b79a]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:19.418198 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.425539 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.007s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:19.425853 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4810212-3eea-4654-b613-882e2e83a954]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:19.426604 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:19.443624 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] CMD "nvme version" returned: 0 in 0.017s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:19.444677 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] <== get_connector_properties: return (75ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:19.526656 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-be73d8a1-2210-4b9c-9c60-e9e3779e5d38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:20.207557 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:13:20.209069 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc991a90>, 'connection_properties': {'name': 'volumes/volume-9723b5a7-1252-49d7-b3f1-ae5426460e67', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '9723b5a7-1252-49d7-b3f1-ae5426460e67', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:20.210241 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:13:20.284205 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] <== connect_volume: return (73ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dcafcca0>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:20.306249 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Starting backup of volume='9723b5a7-1252-49d7-b3f1-ae5426460e67'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:13:20.307844 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Volume file is RBD: attempting optimized backup {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 29 21:13:20.341063 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Creating base image 'volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' {{(pid=134197) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 29 21:13:20.341521 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:13:20.403099 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Using --from-snap 'None' for incremental backup of volume 9723b5a7-1252-49d7-b3f1-ae5426460e67. {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 29 21:13:20.403764 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Creating backup snapshot='backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615' {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 29 21:13:20.956165 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Performing differential transfer from 'volume-9723b5a7-1252-49d7-b3f1-ae5426460e67' to 'volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:13:20.956834 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_3t44r_6m --pool volumes volumes/volume-9723b5a7-1252-49d7-b3f1-ae5426460e67@backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:13:20.958292 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:13:29.010914 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Differential backup transfer completed in 8.0543s {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 29 21:13:29.011542 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:13:29.028900 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:13:29.029328 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:13:29.037765 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:13:29.038738 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:13:29.053300 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}}
Jan 29 21:13:29.054143 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Backing up metadata for volume 9723b5a7-1252-49d7-b3f1-ae5426460e67. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:13:29.131183 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Backup '318bf7e5-1cf6-4993-9ab9-636adc9dda50' of volume 9723b5a7-1252-49d7-b3f1-ae5426460e67 finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:13:29.134198 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc991a90>, 'connection_properties': {'name': 'volumes/volume-9723b5a7-1252-49d7-b3f1-ae5426460e67', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '9723b5a7-1252-49d7-b3f1-ae5426460e67', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dcafcca0>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:29.184637 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] <== disconnect_volume: return (40ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:29.273264 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-2d29c48e-6193-48ab-af52-e93508630b1e tempest-VolumesBackupsTest-2013611801 None] Create backup finished. backup: 318bf7e5-1cf6-4993-9ab9-636adc9dda50.
Jan 29 21:13:31.153374 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Restore backup started, backup: 318bf7e5-1cf6-4993-9ab9-636adc9dda50 volume: 47e1f328-e4d4-4c69-a891-74eeb1d28bdb.
Jan 29 21:13:31.172098 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:31.173875 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.186968 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:31.187898 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[845a1e7e-3b13-4157-ac67-93869ff15c5f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:31.189656 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.198668 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[637622da-2072-45f0-a27c-c67bc7fdd247]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:31.199983 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:31.201762 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[637622da-2072-45f0-a27c-c67bc7fdd247]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:31.203519 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.212170 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d24665a9-27ab-4f4a-9749-5ba1894c494f]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:31.213415 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:31.216232 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[d24665a9-27ab-4f4a-9749-5ba1894c494f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:31.218739 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.233380 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:31.233949 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d5642c6-aecd-4ac3-abc4-2f9f6d74fa9f]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:31.235435 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.245730 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.010s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:31.245962 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[6807f9e1-a98b-4559-98f8-36f000e42b51]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:31.249308 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.256682 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:31.257288 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2b81ca4-e218-4e6f-9ed2-160bc8951b1c]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:31.258283 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:31.276612 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:31.278308 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] <== get_connector_properties: return (104ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:31.347525 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:13:31.348028 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:13:32.108739 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:13:32.111238 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc948130>, 'connection_properties': {'name': 'volumes/volume-47e1f328-e4d4-4c69-a891-74eeb1d28bdb', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '47e1f328-e4d4-4c69-a891-74eeb1d28bdb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:32.112550 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:13:32.234715 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] <== connect_volume: return (122ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca3ebe0>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:32.254731 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:32.274308 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Starting restore from Ceph backup=318bf7e5-1cf6-4993-9ab9-636adc9dda50 to volume=volume-47e1f328-e4d4-4c69-a891-74eeb1d28bdb {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Jan 29 21:13:32.370353 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Looking for snapshot of backup base 'volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:13:32.370991 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Found snapshot 'backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:13:32.390936 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Volume file is RBD. {{(pid=134197) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Jan 29 21:13:32.444828 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Attempting differential restore. {{(pid=134197) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Jan 29 21:13:32.445624 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Attempting incremental restore from base='volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' snap='backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615' {{(pid=134197) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Jan 29 21:13:32.446165 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Performing differential transfer from 'volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' to 'volume-47e1f328-e4d4-4c69-a891-74eeb1d28bdb' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:13:32.446726 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50@backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:13:32.447294 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_5oq5uc0v --pool volumes - volumes/volume-47e1f328-e4d4-4c69-a891-74eeb1d28bdb' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:13:39.071379 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Restore transfer completed in 6.6245s {{(pid=134197) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Jan 29 21:13:39.072932 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Jan 29 21:13:39.102472 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Restoring volume base metadata {{(pid=134197) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Jan 29 21:13:39.127531 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] No metadata of type 'volume-metadata' to restore {{(pid=134197) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 29 21:13:39.128143 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Restoring volume glance metadata {{(pid=134197) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}}
Jan 29 21:13:39.279939 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Restore to volume 47e1f328-e4d4-4c69-a891-74eeb1d28bdb finished successfully. {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Jan 29 21:13:39.283929 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc948130>, 'connection_properties': {'name': 'volumes/volume-47e1f328-e4d4-4c69-a891-74eeb1d28bdb', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '47e1f328-e4d4-4c69-a891-74eeb1d28bdb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca3ebe0>}, 'force': True, ': False}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:39.295346 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] <== disconnect_volume: return (11ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:39.350198 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.manager [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Encryption key ID for volume 47e1f328-e4d4-4c69-a891-74eeb1d28bdb already matches encryption key ID in backup 318bf7e5-1cf6-4993-9ab9-636adc9dda50. {{(pid=134197) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Jan 29 21:13:39.386625 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-bb21a9a0-0265-480e-965b-371304ba242d tempest-VolumesBackupsTest-2013611801 None] Finished restoring backup 318bf7e5-1cf6-4993-9ab9-636adc9dda50 to volume 47e1f328-e4d4-4c69-a891-74eeb1d28bdb.
Jan 29 21:13:40.816332 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Delete backup started, backup: 318bf7e5-1cf6-4993-9ab9-636adc9dda50.
Jan 29 21:13:40.818823 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Delete started for backup=318bf7e5-1cf6-4993-9ab9-636adc9dda50 {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:13:40.819449 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Trying diff format basename='volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' for backup base image of volume 9723b5a7-1252-49d7-b3f1-ae5426460e67. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:13:40.894210 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Looking for snapshot of backup base 'volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:13:40.894710 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Found snapshot 'backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:13:40.895152 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Deleting backup snapshot='backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50.snap.1643490800.4031615' {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 29 21:13:41.056220 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Deleting backup base image='volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' of volume 9723b5a7-1252-49d7-b3f1-ae5426460e67.
Jan 29 21:13:41.565550 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Base backup image='volume-9723b5a7-1252-49d7-b3f1-ae5426460e67.backup.318bf7e5-1cf6-4993-9ab9-636adc9dda50' of volume 9723b5a7-1252-49d7-b3f1-ae5426460e67 deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:13:41.621257 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Delete of backup '318bf7e5-1cf6-4993-9ab9-636adc9dda50' for volume '9723b5a7-1252-49d7-b3f1-ae5426460e67' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:13:41.627617 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:41.660908 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Created reservations ['4f604719-1e1e-4f6c-a71c-f8d2b0e6fbb0', '95cdfd85-00a7-451d-a312-df83e3579b16'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:13:41.692150 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-05066e97-f4e1-4f07-ad3d-e38d7cd0afd9 tempest-VolumesBackupsTest-2013611801 None] Delete backup finished, backup 318bf7e5-1cf6-4993-9ab9-636adc9dda50 deleted.
Jan 29 21:13:48.767923 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Create backup started, backup: 41b5458b-702f-428f-9388-62912dc5e9e0 volume: 92971f7d-729a-472f-a613-4eb09fc0961e.
Jan 29 21:13:48.780748 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:48.782752 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.793751 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:48.794721 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[19815a2b-1afd-46b6-9344-34caee291f0f]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:48.796604 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.804577 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[db680602-714b-4f61-b626-53109fad6716]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:48.805132 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:48.806350 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[db680602-714b-4f61-b626-53109fad6716]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:48.809971 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.816000 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a20ad5a1-5a07-4ac2-a98e-01ff9aa0f6ba]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:13:48.817877 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:13:48.819182 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[a20ad5a1-5a07-4ac2-a98e-01ff9aa0f6ba]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:48.822229 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.842636 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.022s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:48.842965 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[371be680-b9d2-4ddb-8ea3-65a0cf864b84]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:48.845846 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.858674 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.012s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:48.859733 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c8d1490-0bee-4931-bdae-f75945b3fa6e]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:48.861413 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.870266 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.009s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:48.871145 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[112f40fe-cddc-44dd-805a-af16eba37320]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:13:48.871956 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:13:48.891562 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] CMD "nvme version" returned: 0 in 0.019s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:13:48.892751 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] <== get_connector_properties: return (111ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:49.538413 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-be73d8a1-2210-4b9c-9c60-e9e3779e5d38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:49.673554 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:13:49.675153 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc948a00>, 'connection_properties': {'name': 'volumes/volume-92971f7d-729a-472f-a613-4eb09fc0961e', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '92971f7d-729a-472f-a613-4eb09fc0961e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:49.676855 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:13:49.739892 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] <== connect_volume: return (63ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dcbdeaf0>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:49.766955 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Starting backup of volume='92971f7d-729a-472f-a613-4eb09fc0961e'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:13:49.768301 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Volume file is RBD: attempting optimized backup {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 29 21:13:49.803580 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Creating base image 'volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' {{(pid=134197) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 29 21:13:49.803765 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:13:49.855380 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Using --from-snap 'None' for incremental backup of volume 92971f7d-729a-472f-a613-4eb09fc0961e. {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 29 21:13:49.855791 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Creating backup snapshot='backup.41b5458b-702f-428f-9388-62912dc5e9e0.snap.1643490829.8554056' {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 29 21:13:50.736949 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Performing differential transfer from 'volume-92971f7d-729a-472f-a613-4eb09fc0961e' to 'volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:13:50.737443 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_73mo7wpv --pool volumes volumes/volume-92971f7d-729a-472f-a613-4eb09fc0961e@backup.41b5458b-702f-428f-9388-62912dc5e9e0.snap.1643490829.8554056 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:13:50.737891 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:13:51.781307 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Differential backup transfer completed in 1.0450s {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 29 21:13:51.781929 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:13:51.794884 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:13:51.795525 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:13:51.804198 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:13:51.804536 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:13:51.811384 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] No metadata type 'volume-glance-metadata' available {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 29 21:13:51.812005 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Backing up metadata for volume 92971f7d-729a-472f-a613-4eb09fc0961e. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:13:51.874679 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Backup '41b5458b-702f-428f-9388-62912dc5e9e0' of volume 92971f7d-729a-472f-a613-4eb09fc0961e finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:13:51.876960 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc948a00>, 'connection_properties': {'name': 'volumes/volume-92971f7d-729a-472f-a613-4eb09fc0961e', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '92971f7d-729a-472f-a613-4eb09fc0961e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dcbdeaf0>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:13:51.907816 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] <== disconnect_volume: return (30ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:13:52.041541 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-9122f769-939c-4fa9-a9e2-02ce59c99c41 tempest-VolumesBackupsV39Test-648724410 None] Create backup finished. backup: 41b5458b-702f-428f-9388-62912dc5e9e0.
Jan 29 21:13:53.021944 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Delete backup started, backup: 41b5458b-702f-428f-9388-62912dc5e9e0.
Jan 29 21:13:53.029209 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Delete started for backup=41b5458b-702f-428f-9388-62912dc5e9e0 {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:13:53.029862 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Trying diff format basename='volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' for backup base image of volume 92971f7d-729a-472f-a613-4eb09fc0961e. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:13:53.083437 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Looking for snapshot of backup base 'volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:13:53.083822 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Found snapshot 'backup.41b5458b-702f-428f-9388-62912dc5e9e0.snap.1643490829.8554056' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:13:53.084192 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Deleting backup snapshot='backup.41b5458b-702f-428f-9388-62912dc5e9e0.snap.1643490829.8554056' {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 29 21:13:53.786778 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Deleting backup base image='volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' of volume 92971f7d-729a-472f-a613-4eb09fc0961e.
Jan 29 21:13:53.929660 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Base backup image='volume-92971f7d-729a-472f-a613-4eb09fc0961e.backup.41b5458b-702f-428f-9388-62912dc5e9e0' of volume 92971f7d-729a-472f-a613-4eb09fc0961e deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:13:53.962032 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Delete of backup '41b5458b-702f-428f-9388-62912dc5e9e0' for volume '92971f7d-729a-472f-a613-4eb09fc0961e' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:13:53.968713 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:13:53.991261 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Created reservations ['a52181e8-9d91-4588-9d35-34dd497a054d', '4c071048-597a-4843-a157-c43772c0c891'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:13:54.014293 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-4903320e-7d13-4652-923a-5f4afd578dcf tempest-VolumesBackupsV39Test-648724410 None] Delete backup finished, backup 41b5458b-702f-428f-9388-62912dc5e9e0 deleted.
Jan 29 21:14:13.245161 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Create backup started, backup: 3f51c3c7-6180-4fe7-a5af-64c3c501f14e volume: d8cd7d9c-a78d-49fb-90ce-a7c609636f75.
Jan 29 21:14:13.252838 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:13.254641 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.261682 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:13.262040 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5476297-76b7-4769-b295-6b64a8a4c602]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:13.263418 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.269975 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[7b1783b0-e7ad-4c1c-9a9b-3039d64d94a3]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:14:13.271086 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:14:13.272638 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b1783b0-e7ad-4c1c-9a9b-3039d64d94a3]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:13.275260 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.279121 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d2834a8b-ee80-404f-a8a7-2558b7e60690]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:14:13.279935 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:14:13.281457 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2834a8b-ee80-404f-a8a7-2558b7e60690]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:13.282217 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.292092 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.010s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:13.292611 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8c15077-7afe-4c21-b513-3418bec2a706]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:13.295631 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.303319 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:13.303726 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca1f3ca4-5221-4558-b38c-8b81d297d245]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:13.307183 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.313402 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.008s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:13.313801 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[820d9f3f-a083-4cb7-9556-224c6b4471a9]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:13.314674 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:13.332279 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] CMD "nvme version" returned: 0 in 0.017s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:13.333939 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] <== get_connector_properties: return (80ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:14.119027 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:14:14.120543 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc038be0>, 'connection_properties': {'name': 'volumes/volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'd8cd7d9c-a78d-49fb-90ce-a7c609636f75', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:14.121872 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:14:14.161902 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] <== connect_volume: return (40ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc036b50>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:14.170426 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:14.189629 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Starting backup of volume='d8cd7d9c-a78d-49fb-90ce-a7c609636f75'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:14:14.190571 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Volume file is RBD: attempting optimized backup {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 29 21:14:14.211161 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Creating base image 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' {{(pid=134197) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 29 21:14:14.211568 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:14:14.238369 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Using --from-snap 'None' for incremental backup of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75. {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 29 21:14:14.239290 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Creating backup snapshot='backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952' {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 29 21:14:14.986749 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Performing differential transfer from 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75' to 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:14:14.987319 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_so6yq5m0 --pool volumes volumes/volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75@backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:14:14.987674 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:14:16.015148 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Differential backup transfer completed in 1.0279s {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 29 21:14:16.015820 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:14:16.030579 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:14:16.030985 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:14:16.037252 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:14:16.037638 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:14:16.043531 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] No metadata type 'volume-glance-metadata' available {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 29 21:14:16.044058 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Backing up metadata for volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:14:16.074238 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Backup '3f51c3c7-6180-4fe7-a5af-64c3c501f14e' of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75 finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:14:16.075911 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc038be0>, 'connection_properties': {'name': 'volumes/volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': 'd8cd7d9c-a78d-49fb-90ce-a7c609636f75', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc036b50>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:16.090245 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] <== disconnect_volume: return (14ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:16.142592 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-ccbd0e26-ed1b-4034-8103-a1d9e00fbf58 tempest-VolumesBackupsAdminTest-936429362 None] Create backup finished. backup: 3f51c3c7-6180-4fe7-a5af-64c3c501f14e.
Jan 29 21:14:16.518816 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-f7d9b0d7-1f4a-418e-bf79-194f17700d21 tempest-VolumesBackupsAdminTest-846986595 None] Export record started, backup: 3f51c3c7-6180-4fe7-a5af-64c3c501f14e.
Jan 29 21:14:16.520727 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-f7d9b0d7-1f4a-418e-bf79-194f17700d21 tempest-VolumesBackupsAdminTest-846986595 None] Export record finished, backup 3f51c3c7-6180-4fe7-a5af-64c3c501f14e exported.
Jan 29 21:14:16.661929 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-4b183192-825c-4072-a3c6-22fe38bf7caf tempest-VolumesBackupsAdminTest-846986595 None] Import record started, backup_url: eyJkcml2ZXJfaW5mbyI6IHt9LCAiaWQiOiAiNmI0MzMwYzEtMTk0OS00NDQxLWFhOWUtYTdmYzFmZWQ4YTJiIiwgInVzZXJfaWQiOiAiYWRmZWJmM2MyNGE3NGQ3M2JkMjQ5YTUyYzBmYTBmMTgiLCAicHJvamVjdF9pZCI6ICJkNjMzMzgyNTE4MDE0NWRhOTJjNGRhYjQ5YmFjMDQ0NiIsICJ2b2x1bWVfaWQiOiAiZDhjZDdkOWMtYTc4ZC00OWZiLTkwY2UtYTdjNjA5NjM2Zjc1IiwgImhvc3QiOiAibmNkdi1tYXN0ZXItMSIsICJhdmFpbGFiaWxpdHlfem9uZSI6IG51bGwsICJjb250YWluZXIiOiAiYmFja3VwcyIsICJwYXJlbnRfaWQiOiBudWxsLCAic3RhdHVzIjogImF2YWlsYWJsZSIsICJmYWlsX3JlYXNvbiI6IG51bGwsICJzaXplIjogMSwgImRpc3BsYXlfbmFtZSI6ICJ0ZW1wZXN0LVZvbHVtZXNCYWNrdXBzQWRtaW5UZXN0LUJhY2t1cC0xMTcwODY1MjIzIiwgImRpc3BsYXlfZGVzY3JpcHRpb24iOiBudWxsLCAic2VydmljZV9tZXRhZGF0YSI6ICJ7XCJiYXNlXCI6IFwidm9sdW1lLWQ4Y2Q3ZDljLWE3OGQtNDlmYi05MGNlLWE3YzYwOTYzNmY3NS5iYWNrdXAuM2Y1MWMzYzctNjE4MC00ZmU3LWE1YWYtNjRjM2M1MDFmMTRlXCJ9IiwgInNlcnZpY2UiOiAiY2luZGVyLmJhY2t1cC5kcml2ZXJzLmNlcGguQ2VwaEJhY2t1cERyaXZlciIsICJvYmplY3RfY291bnQiOiAwLCAidGVtcF92b2x1bWVfaWQiOiBudWxsLCAidGVtcF9zbmFwc2hvdF9pZCI6IG51bGwsICJudW1fZGVwZW5kZW50X2JhY2t1cHMiOiAwLCAic25hcHNob3RfaWQiOiBudWxsLCAiZGF0YV90aW1lc3RhbXAiOiAiMjAyMi0wMS0yOVQyMToxNDoxM1oiLCAicmVzdG9yZV92b2x1bWVfaWQiOiBudWxsLCAibWV0YWRhdGEiOiB7fSwgImVuY3J5cHRpb25fa2V5X2lkIjogbnVsbCwgImNyZWF0ZWRfYXQiOiAiMjAyMi0wMS0yOVQyMToxNDoxM1oiLCAidXBkYXRlZF9hdCI6ICIyMDIyLTAxLTI5VDIxOjE0OjE2WiIsICJkZWxldGVkX2F0IjogbnVsbCwgImRlbGV0ZWQiOiBmYWxzZX0=.
Jan 29 21:14:16.669638 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-4b183192-825c-4072-a3c6-22fe38bf7caf tempest-VolumesBackupsAdminTest-846986595 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:16.709547 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-4b183192-825c-4072-a3c6-22fe38bf7caf tempest-VolumesBackupsAdminTest-846986595 None] Import record id 6b4330c1-1949-4441-aa9e-a7fc1fed8a2b metadata from driver finished.
Jan 29 21:14:19.089377 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Restore backup started, backup: 3f51c3c7-6180-4fe7-a5af-64c3c501f14e volume: 93eb89bc-d004-4e77-9222-3e4292575b20.
Jan 29 21:14:19.103331 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:19.104743 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.112637 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:19.113050 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[94fdca60-dd1e-41fc-9ac5-2b94b44ed82c]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:19.114373 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.119954 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ebf18010-8345-4d11-b476-bbf420caca69]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:14:19.120573 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:14:19.122041 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebf18010-8345-4d11-b476-bbf420caca69]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:19.122308 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.127460 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[85fd8232-d3d0-48a9-9bd3-3dc13dd79f2a]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:14:19.128004 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:14:19.129927 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[85fd8232-d3d0-48a9-9bd3-3dc13dd79f2a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:19.130255 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.138116 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:19.138436 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8d92400-8eaf-4bc8-8d60-a96930268ec4]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:19.139527 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.148939 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.009s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:19.149435 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[432b08c3-4803-4f43-8a83-9504a58f0a8c]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:19.150943 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.157278 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:19.157607 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccda7ee2-198d-4037-8a92-2a9a7939002b]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:19.158804 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:19.177688 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:19.178745 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] <== get_connector_properties: return (74ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:19.538748 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-be73d8a1-2210-4b9c-9c60-e9e3779e5d38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:19.865745 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:14:19.867209 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dca04b80>, 'connection_properties': {'name': 'volumes/volume-93eb89bc-d004-4e77-9222-3e4292575b20', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '93eb89bc-d004-4e77-9222-3e4292575b20', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:19.868752 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:14:19.907754 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] <== connect_volume: return (39ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc05aee0>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:19.920835 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Starting restore from Ceph backup=3f51c3c7-6180-4fe7-a5af-64c3c501f14e to volume=volume-93eb89bc-d004-4e77-9222-3e4292575b20 {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1274}}
Jan 29 21:14:19.984857 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Looking for snapshot of backup base 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:14:19.985292 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Found snapshot 'backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:14:19.992178 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Volume file is RBD. {{(pid=134197) _diff_restore_allowed /opt/stack/cinder/cinder/backup/drivers/ceph.py:1184}}
Jan 29 21:14:20.034369 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Attempting differential restore. {{(pid=134197) _restore_volume /opt/stack/cinder/cinder/backup/drivers/ceph.py:1234}}
Jan 29 21:14:20.035122 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Attempting incremental restore from base='volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' snap='backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952' {{(pid=134197) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1089}}
Jan 29 21:14:20.035238 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Performing differential transfer from 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' to 'volume-93eb89bc-d004-4e77-9222-3e4292575b20' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:14:20.036213 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Piping cmd1='rbd export-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups backups/volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e@backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:14:20.036583 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] cmd2='rbd import-diff --id cinder --conf /tmp/brickrbd_c6eauf8w --pool volumes - volumes/volume-93eb89bc-d004-4e77-9222-3e4292575b20' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:14:21.097600 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Restore transfer completed in 1.0617s {{(pid=134197) _diff_restore_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:1111}}
Jan 29 21:14:21.098630 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Restore target I/O object does not support fileno() - skipping call to fsync(). {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1285}}
Jan 29 21:14:21.123075 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Restoring volume base metadata {{(pid=134197) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}}
Jan 29 21:14:21.150456 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] No metadata of type 'volume-metadata' to restore {{(pid=134197) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 29 21:14:21.150898 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=134197) put /opt/stack/cinder/cinder/backup/driver.py:345}}
Jan 29 21:14:21.157904 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Restore to volume 93eb89bc-d004-4e77-9222-3e4292575b20 finished successfully. {{(pid=134197) restore /opt/stack/cinder/cinder/backup/drivers/ceph.py:1292}}
Jan 29 21:14:21.159700 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dca04b80>, 'connection_properties': {'name': 'volumes/volume-93eb89bc-d004-4e77-9222-3e4292575b20', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '93eb89bc-d004-4e77-9222-3e4292575b20', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc05aee0>}, 'force': True, ': False}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:21.187363 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] <== disconnect_volume: return (27ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:21.240572 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.manager [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Encryption key ID for volume 93eb89bc-d004-4e77-9222-3e4292575b20 already matches encryption key ID in backup 3f51c3c7-6180-4fe7-a5af-64c3c501f14e. {{(pid=134197) _run_restore /opt/stack/cinder/cinder/backup/manager.py:801}}
Jan 29 21:14:21.271914 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-ebdd4af3-347b-4595-9e4c-f5e1cdf7d92f tempest-VolumesBackupsAdminTest-936429362 None] Finished restoring backup 3f51c3c7-6180-4fe7-a5af-64c3c501f14e to volume 93eb89bc-d004-4e77-9222-3e4292575b20.
Jan 29 21:14:22.417839 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Delete backup started, backup: 6b4330c1-1949-4441-aa9e-a7fc1fed8a2b.
Jan 29 21:14:22.422660 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Delete started for backup=6b4330c1-1949-4441-aa9e-a7fc1fed8a2b {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:14:22.423651 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Trying diff format basename='volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' for backup base image of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:14:22.484921 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Looking for snapshot of backup base 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:14:22.485580 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Backup '6b4330c1-1949-4441-aa9e-a7fc1fed8a2b' has no snapshot {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:915}}
Jan 29 21:14:22.486104 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] No backup snapshot to delete {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Jan 29 21:14:22.495359 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Backup base image of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75 still has 1 snapshots so skipping base image delete.
Jan 29 21:14:22.528033 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Metadata backup object 'backup.6b4330c1-1949-4441-aa9e-a7fc1fed8a2b.meta' not found - ignoring {{(pid=134197) remove_if_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:160}}
Jan 29 21:14:22.531286 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Delete of backup '6b4330c1-1949-4441-aa9e-a7fc1fed8a2b' for volume 'd8cd7d9c-a78d-49fb-90ce-a7c609636f75' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:14:22.538243 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:22.565765 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Created reservations ['3d162d86-133e-4df3-9e49-dff030625c50', '3ca76ecd-27a3-4370-9c01-58d2fa5b0403'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:14:22.600857 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-502eebfe-3af7-4837-9f06-3f7c69d86765 tempest-VolumesBackupsAdminTest-846986595 None] Delete backup finished, backup 6b4330c1-1949-4441-aa9e-a7fc1fed8a2b deleted.
Jan 29 21:14:23.497819 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Delete backup started, backup: 3f51c3c7-6180-4fe7-a5af-64c3c501f14e.
Jan 29 21:14:23.499330 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Delete started for backup=3f51c3c7-6180-4fe7-a5af-64c3c501f14e {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:14:23.499847 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Trying diff format basename='volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' for backup base image of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:14:23.557101 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Looking for snapshot of backup base 'volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:14:23.557487 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Found snapshot 'backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:14:23.557884 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Deleting backup snapshot='backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e.snap.1643490854.2383952' {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 29 21:14:24.113252 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Deleting backup base image='volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75.
Jan 29 21:14:24.264979 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Base backup image='volume-d8cd7d9c-a78d-49fb-90ce-a7c609636f75.backup.3f51c3c7-6180-4fe7-a5af-64c3c501f14e' of volume d8cd7d9c-a78d-49fb-90ce-a7c609636f75 deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:14:24.309955 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Delete of backup '3f51c3c7-6180-4fe7-a5af-64c3c501f14e' for volume 'd8cd7d9c-a78d-49fb-90ce-a7c609636f75' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:14:24.315596 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:24.339110 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Created reservations ['9b1a47a4-25d4-41a4-af42-139fcceca641', 'cf138022-2c1e-4ec6-9a50-fdf061055b4e'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:14:24.363265 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-e597bd90-7928-4c5a-b3cd-cbb4800b811b tempest-VolumesBackupsAdminTest-936429362 None] Delete backup finished, backup 3f51c3c7-6180-4fe7-a5af-64c3c501f14e deleted.
Jan 29 21:14:25.986837 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Create backup started, backup: 49f039ad-c2a2-48cd-933d-e6db5dbe9db4 volume: 87559b37-b61d-4d6e-bf95-537f1fe13f01.
Jan 29 21:14:25.997581 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:25.999218 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.015238 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:26.015778 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[f66aa8b9-2bfb-4be6-a1e9-0fcd71ea2e34]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:26.020456 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.026859 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ccd2c86e-2dc1-4371-8952-1ceae1058251]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:14:26.027962 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:14:26.029528 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccd2c86e-2dc1-4371-8952-1ceae1058251]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:26.033623 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.042458 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[49a19c59-6cd7-4d60-bf8a-249cf13785c8]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:14:26.043488 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:14:26.044842 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[49a19c59-6cd7-4d60-bf8a-249cf13785c8]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:26.048650 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.060714 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.014s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:26.061309 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[72de2f87-9540-4928-aec5-76f7a2caeab5]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:26.065989 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.084415 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.011s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:26.084415 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e6b5ba1-6769-48df-bb3f-3aa1c3351c8b]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:26.089435 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.098595 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.011s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:26.099041 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[05953453-bd7a-4b76-9493-f8dbf628f717]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:14:26.099807 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:14:26.133181 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] CMD "nvme version" returned: 0 in 0.033s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:14:26.134329 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] <== get_connector_properties: return (136ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:26.827964 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:14:26.829454 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dcb7b700>, 'connection_properties': {'name': 'volumes/volume-87559b37-b61d-4d6e-bf95-537f1fe13f01', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '87559b37-b61d-4d6e-bf95-537f1fe13f01', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:26.830508 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:14:26.868541 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] <== connect_volume: return (38ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc947f40>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:26.875999 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:26.893245 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Starting backup of volume='87559b37-b61d-4d6e-bf95-537f1fe13f01'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:14:26.894212 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Volume file is RBD: attempting optimized backup {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:985}}
Jan 29 21:14:26.915055 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Creating base image 'volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' {{(pid=134197) _create_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:444}}
Jan 29 21:14:26.915461 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:14:26.940625 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Using --from-snap 'None' for incremental backup of volume 87559b37-b61d-4d6e-bf95-537f1fe13f01. {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:764}}
Jan 29 21:14:26.941086 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Creating backup snapshot='backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4.snap.1643490866.9406662' {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:770}}
Jan 29 21:14:27.983149 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Performing differential transfer from 'volume-87559b37-b61d-4d6e-bf95-537f1fe13f01' to 'volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' {{(pid=134197) _rbd_diff_transfer /opt/stack/cinder/cinder/backup/drivers/ceph.py:624}}
Jan 29 21:14:27.983856 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Piping cmd1='rbd export-diff --id cinder --conf /tmp/brickrbd_tr85uy5j --pool volumes volumes/volume-87559b37-b61d-4d6e-bf95-537f1fe13f01@backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4.snap.1643490866.9406662 -' into... {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:585}}
Jan 29 21:14:27.984118 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] cmd2='rbd import-diff --id cinder-bak --conf /etc/ceph/ceph.conf --pool backups - backups/volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' {{(pid=134197) _piped_execute /opt/stack/cinder/cinder/backup/drivers/ceph.py:586}}
Jan 29 21:14:29.131236 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Differential backup transfer completed in 1.1476s {{(pid=134197) _backup_rbd /opt/stack/cinder/cinder/backup/drivers/ceph.py:790}}
Jan 29 21:14:29.131871 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:14:29.144374 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:14:29.145247 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:14:29.152550 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:14:29.152925 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:14:29.159121 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] No metadata type 'volume-glance-metadata' available {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 29 21:14:29.159671 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Backing up metadata for volume 87559b37-b61d-4d6e-bf95-537f1fe13f01. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:14:29.201083 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Backup '49f039ad-c2a2-48cd-933d-e6db5dbe9db4' of volume 87559b37-b61d-4d6e-bf95-537f1fe13f01 finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:14:29.202768 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dcb7b700>, 'connection_properties': {'name': 'volumes/volume-87559b37-b61d-4d6e-bf95-537f1fe13f01', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '87559b37-b61d-4d6e-bf95-537f1fe13f01', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dc947f40>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:14:29.218596 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] <== disconnect_volume: return (15ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:14:29.281824 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-a922e704-a9f1-4e80-b3ce-3494f29e6f21 tempest-VolumesBackupsAdminTest-936429362 None] Create backup finished. backup: 49f039ad-c2a2-48cd-933d-e6db5dbe9db4.
Jan 29 21:14:30.190342 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-69aef1e9-4b0c-4416-8241-842b261ea81a tempest-VolumesBackupsAdminTest-846986595 None] Reset backup status started, backup_id: 49f039ad-c2a2-48cd-933d-e6db5dbe9db4, status: error.
Jan 29 21:14:30.190881 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-69aef1e9-4b0c-4416-8241-842b261ea81a tempest-VolumesBackupsAdminTest-846986595 None] Backup service: cinder.backup.drivers.ceph.CephBackupDriver.
Jan 29 21:14:30.264299 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Delete backup started, backup: 49f039ad-c2a2-48cd-933d-e6db5dbe9db4.
Jan 29 21:14:30.266064 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Delete started for backup=49f039ad-c2a2-48cd-933d-e6db5dbe9db4 {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:14:30.266596 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Trying diff format basename='volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' for backup base image of volume 87559b37-b61d-4d6e-bf95-537f1fe13f01. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:14:30.321823 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Looking for snapshot of backup base 'volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:14:30.322286 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Found snapshot 'backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4.snap.1643490866.9406662' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:924}}
Jan 29 21:14:30.322751 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Deleting backup snapshot='backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4.snap.1643490866.9406662' {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:473}}
Jan 29 21:14:31.169127 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Deleting backup base image='volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' of volume 87559b37-b61d-4d6e-bf95-537f1fe13f01.
Jan 29 21:14:31.330866 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Base backup image='volume-87559b37-b61d-4d6e-bf95-537f1fe13f01.backup.49f039ad-c2a2-48cd-933d-e6db5dbe9db4' of volume 87559b37-b61d-4d6e-bf95-537f1fe13f01 deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:14:31.348672 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:14:31.349146 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:14:31.369582 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Delete of backup '49f039ad-c2a2-48cd-933d-e6db5dbe9db4' for volume '87559b37-b61d-4d6e-bf95-537f1fe13f01' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:14:31.375815 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:14:31.399585 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Created reservations ['44a2203d-d3f3-4d37-98ce-1320030ad0a7', '2ce7c168-f618-44ca-8a7d-0e9dc2fb6dcd'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:14:31.426049 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-51aa5ba2-2fc8-4e8d-bd34-8178968334fb tempest-VolumesBackupsAdminTest-936429362 None] Delete backup finished, backup 49f039ad-c2a2-48cd-933d-e6db5dbe9db4 deleted.
Jan 29 21:15:31.344782 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:15:31.345514 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:16:31.350915 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:16:31.351944 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:17:31.355968 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:17:31.357827 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:18:31.360010 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:18:31.360670 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:19:31.356625 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:19:31.357355 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:20:31.361741 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:20:31.362360 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:21:31.367478 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:21:31.368056 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:22:31.372499 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:22:31.373157 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:23:31.377897 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:23:31.378542 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:24:31.382965 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:24:31.383509 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:25:31.388377 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:25:31.389610 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:26:31.393561 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:26:31.394124 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:27:31.389348 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:27:31.389962 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:28:31.395327 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:28:31.395931 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:29:31.400432 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:29:31.401047 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:30:31.405650 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:30:31.406244 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:31:31.410468 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:31:31.411048 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:31:47.194127 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Create backup started, backup: 5d9195db-2db0-495b-b8c2-9f0eebd3b40a volume: 5e2560b7-72da-4486-9079-99b40ddbdacd snapshot: a129b379-fcaa-45f7-816a-27e0191ce41c.
Jan 29 21:31:47.202476 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '10.222.0.20', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:31:47.204467 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.212087 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:31:47.212456 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ce74de5-f4f9-432c-8c22-35f054be6019]: (4, ('## DO NOT EDIT OR REMOVE THIS FILE!\n## If you remove this file, the iSCSI daemon will not start.\n## If you change the InitiatorName, existing access control lists\n## may reject this initiator.  The InitiatorName must be unique\n## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.\nInitiatorName=iqn.1993-08.org.debian:01:9aa44ce77761\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:31:47.213747 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.220048 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[44c0a3cb-b2bf-4105-b877-8d7d6d303991]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:31:47.221014 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:31:47.222391 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[44c0a3cb-b2bf-4105-b877-8d7d6d303991]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:31:47.224830 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): systool -c fc_host -v {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.228262 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] 'systool -c fc_host -v' failed. Not Retrying. {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:473}}
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b235870c-f633-4206-ad31-d685b6684e6f]: [Errno 2] No such file or directory: 'systool' {{(pid=160493) _process_cmd /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:480}}
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]: Traceback (most recent call last):
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 477, in _process_cmd
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     ret = func(*f_args, **f_kwargs)
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 274, in _wrap
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     return func(*args, **kwargs)
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 197, in execute_root
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/os_brick/privileged/rootwrap.py", line 145, in custom_execute
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     return putils.execute(on_execute=on_execute,
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 395, in execute
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     obj = subprocess.Popen(cmd,
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     self._execute_child(args, executable, preexec_fn, close_fds,
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:   File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]:     raise child_exception_type(errno_num, err_msg, err_filename)
Jan 29 21:31:47.228779 ncdv-master-1 cinder-backup[134197]: FileNotFoundError: [Errno 2] No such file or directory: 'systool'
Jan 29 21:31:47.230097 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[b235870c-f633-4206-ad31-d685b6684e6f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:31:47.230392 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt / -n -o SOURCE {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.238303 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt / -n -o SOURCE" returned: 0 in 0.008s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:31:47.238623 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5e9c664-ab3d-4ffc-bd67-8897dac4c5e9]: (4, ('/dev/sda1\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:31:47.239688 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/sda1 -s UUID -o value {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.247579 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/sda1 -s UUID -o value" returned: 0 in 0.008s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:31:47.247931 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1020326-6053-48e2-be3f-2b24616c618a]: (4, ('02bb033c-a191-40c0-94fb-bb025c724b13\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:31:47.249120 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /sys/class/dmi/id/product_uuid {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.255734 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [-] CMD "cat /sys/class/dmi/id/product_uuid" returned: 0 in 0.006s {{(pid=160493) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:31:47.256053 ncdv-master-1 cinder-backup[134197]: DEBUG oslo.privsep.daemon [-] privsep: reply[d028a035-6ec1-43f8-afbd-f98b4ebaf8ba]: (4, ('71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4\n', '')) {{(pid=160493) _call_back /usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py:502}}
Jan 29 21:31:47.256774 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Running cmd (subprocess): nvme version {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jan 29 21:31:47.275455 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_concurrency.processutils [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] CMD "nvme version" returned: 0 in 0.018s {{(pid=134197) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jan 29 21:31:47.276518 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.utils [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] <== get_connector_properties: return (73ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.222.0.20', 'host': 'ncdv-master-1', 'multipath': False, 'initiator': 'iqn.1993-08.org.debian:01:9aa44ce77761', 'do_local_attach': False, 'uuid': '02bb033c-a191-40c0-94fb-bb025c724b13', 'system uuid': '71f5cc74-3c2f-c143-94cc-7b1fc9a7b8b4', 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:5a05b4dc-7eac-41ef-a8a1-5dc1bafd8e4b'} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:31:48.140863 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connector [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Factory for rbd on None {{(pid=134197) factory /usr/local/lib/python3.8/dist-packages/os_brick/initiator/connector.py:281}}
Jan 29 21:31:48.142384 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc95aac0>, 'connection_properties': {'name': 'volumes/volume-851b7de6-49f5-4b68-9a7e-d8b7e8257d65', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '851b7de6-49f5-4b68-9a7e-d8b7e8257d65', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, ': False}}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:31:48.143436 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.linuxrbd [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] opening connection to ceph cluster (timeout=-1). {{(pid=134197) connect /usr/local/lib/python3.8/dist-packages/os_brick/initiator/linuxrbd.py:70}}
Jan 29 21:31:48.209382 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] <== connect_volume: return (66ms) {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca04d30>} {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:31:48.217261 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:31:48.234418 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Starting backup of volume='5e2560b7-72da-4486-9079-99b40ddbdacd'. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:974}}
Jan 29 21:31:48.259248 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] RBD also supports fast-diff, enabling it together with exclusive-lock and object-map {{(pid=134197) _get_rbd_support /opt/stack/cinder/cinder/backup/drivers/ceph.py:272}}
Jan 29 21:31:48.259687 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Creating backup base image='volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.base' for volume 5e2560b7-72da-4486-9079-99b40ddbdacd. {{(pid=134197) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:830}}
Jan 29 21:31:48.288300 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Copying data from volume 5e2560b7-72da-4486-9079-99b40ddbdacd. {{(pid=134197) _full_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:842}}
Jan 29 21:31:48.305876 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferring data between 'volume-5e2560b7-72da-4486-9079-99b40ddbdacd' and 'volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.base' {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:399}}
Jan 29 21:31:48.306253 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] 8 chunks of 134217728 bytes to be transferred {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:403}}
Jan 29 21:31:49.273760 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 1 of 8 (135563K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:50.220546 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 2 of 8 (138534K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:51.170001 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 3 of 8 (138137K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:52.140076 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 4 of 8 (135202K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:53.142613 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 5 of 8 (130833K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:54.137774 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 6 of 8 (131788K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:55.106969 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 7 of 8 (135696K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:56.088300 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Transferred chunk 8 of 8 (133644K/s) {{(pid=134197) _transfer_data /opt/stack/cinder/cinder/backup/drivers/ceph.py:422}}
Jan 29 21:31:56.128322 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Getting metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}}
Jan 29 21:31:56.138955 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=134197) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}}
Jan 29 21:31:56.139352 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Getting metadata type 'volume-metadata' {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}}
Jan 29 21:31:56.145344 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] No metadata type 'volume-metadata' available {{(pid=134197) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}}
Jan 29 21:31:56.145729 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Getting metadata type 'volume-glance-metadata' {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}}
Jan 29 21:31:56.151579 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.driver [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] No metadata type 'volume-glance-metadata' available {{(pid=134197) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}}
Jan 29 21:31:56.152115 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Backing up metadata for volume 5e2560b7-72da-4486-9079-99b40ddbdacd. {{(pid=134197) _backup_metadata /opt/stack/cinder/cinder/backup/drivers/ceph.py:951}}
Jan 29 21:31:56.182615 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Backup '5d9195db-2db0-495b-b8c2-9f0eebd3b40a' of volume 5e2560b7-72da-4486-9079-99b40ddbdacd finished. {{(pid=134197) backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1015}}
Jan 29 21:31:56.184000 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] ==> disconnect_volume: call "{'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7ff6dc95aac0>, 'connection_properties': {'name': 'volumes/volume-851b7de6-49f5-4b68-9a7e-d8b7e8257d65', 'hosts': ['10.222.0.20'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '***', 'volume_id': '851b7de6-49f5-4b68-9a7e-d8b7e8257d65', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7ff6dca04d30>}, 'force': True, ': True}" {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:157}}
Jan 29 21:31:56.192700 ncdv-master-1 cinder-backup[134197]: DEBUG os_brick.initiator.connectors.rbd [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] <== disconnect_volume: return (7ms) None {{(pid=134197) trace_logging_wrapper /usr/local/lib/python3.8/dist-packages/os_brick/utils.py:184}}
Jan 29 21:31:56.264831 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-52de1e11-d0df-463f-8943-5d82aae6e76e tempest-VolumesSnapshotTestJSON-16764377 None] Create backup finished. backup: 5d9195db-2db0-495b-b8c2-9f0eebd3b40a.
Jan 29 21:31:56.488221 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Delete backup started, backup: 5d9195db-2db0-495b-b8c2-9f0eebd3b40a.
Jan 29 21:31:56.490037 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Delete started for backup=5d9195db-2db0-495b-b8c2-9f0eebd3b40a {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1301}}
Jan 29 21:31:56.490486 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Trying diff format basename='volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.5d9195db-2db0-495b-b8c2-9f0eebd3b40a' for backup base image of volume 5e2560b7-72da-4486-9079-99b40ddbdacd. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:513}}
Jan 29 21:31:56.519152 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Image 'volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.5d9195db-2db0-495b-b8c2-9f0eebd3b40a' not found - trying diff format name {{(pid=134197) _rbd_image_exists /opt/stack/cinder/cinder/backup/drivers/ceph.py:661}}
Jan 29 21:31:56.538886 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Looking for snapshot of backup base 'volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.base' {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:905}}
Jan 29 21:31:56.539271 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Backup base 'volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.base' has no snapshots {{(pid=134197) _get_backup_snap_name /opt/stack/cinder/cinder/backup/drivers/ceph.py:908}}
Jan 29 21:31:56.539643 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] No backup snapshot to delete {{(pid=134197) _delete_backup_snapshot /opt/stack/cinder/cinder/backup/drivers/ceph.py:476}}
Jan 29 21:31:56.545088 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Deleting backup base image='volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.base' of volume 5e2560b7-72da-4486-9079-99b40ddbdacd.
Jan 29 21:31:57.030147 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Base backup image='volume-5e2560b7-72da-4486-9079-99b40ddbdacd.backup.base' of volume 5e2560b7-72da-4486-9079-99b40ddbdacd deleted. {{(pid=134197) _try_delete_base_image /opt/stack/cinder/cinder/backup/drivers/ceph.py:561}}
Jan 29 21:31:57.063288 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.backup.drivers.ceph [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Delete of backup '5d9195db-2db0-495b-b8c2-9f0eebd3b40a' for volume '5e2560b7-72da-4486-9079-99b40ddbdacd' finished. {{(pid=134197) delete_backup /opt/stack/cinder/cinder/backup/drivers/ceph.py:1329}}
Jan 29 21:31:57.069389 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_db.sqlalchemy.engines [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=134197) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jan 29 21:31:57.092359 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.quota [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Created reservations ['201cb1a4-1a89-4b1d-a4ad-0d322c9e9ead', '0d237e1d-f6d9-4339-8f0d-5fab9852e6bc'] {{(pid=134197) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jan 29 21:31:57.114536 ncdv-master-1 cinder-backup[134197]: INFO cinder.backup.manager [None req-06f08391-3b87-498f-8d07-d11cc5436c27 tempest-VolumesSnapshotTestJSON-16764377 None] Delete backup finished, backup 5d9195db-2db0-495b-b8c2-9f0eebd3b40a deleted.
Jan 29 21:32:31.415665 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:32:31.416223 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:33:31.420912 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:33:31.421634 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:34:31.425304 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:34:31.425930 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:35:31.429774 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:35:31.430355 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:36:31.434483 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:36:31.435611 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:37:31.431197 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:37:31.431853 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:38:31.436210 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:38:31.436907 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:39:31.441518 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:39:31.442130 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:40:31.437290 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:40:31.437919 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:41:31.442726 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:41:31.443389 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:43:31.440438 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:43:31.441059 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:44:31.445732 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:44:31.446390 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:45:31.451633 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:45:31.452234 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:46:31.457190 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:46:31.457760 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:47:31.462485 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:47:31.463065 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:48:31.467077 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:48:31.467663 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:49:31.471990 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:49:31.472551 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:50:31.469806 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:50:31.470376 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:51:31.475064 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:51:31.475709 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:52:31.480096 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:52:31.480640 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:53:31.485249 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:53:31.485880 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:54:31.490492 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:54:31.491128 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:55:31.495898 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:55:31.496555 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:56:31.500365 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:56:31.501055 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:57:31.505595 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:57:31.507312 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:58:31.501586 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:58:31.502219 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 21:59:31.506879 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 21:59:31.507573 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:00:31.511985 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:00:31.512636 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:01:31.517071 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:01:31.517724 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:02:31.522326 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:02:31.522990 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:03:31.527862 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:03:31.528570 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:04:31.533069 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:04:31.533749 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:05:31.537891 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:05:31.538543 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:06:31.533655 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:06:31.534333 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:07:31.538817 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:07:31.539524 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:08:31.536176 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:08:31.536836 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:09:31.531901 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:09:31.532575 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:10:31.532441 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:10:31.533117 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:11:31.537762 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:11:31.538414 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:12:31.542680 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:12:31.543367 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:13:31.538550 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:13:31.539216 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:14:31.543915 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:14:31.544579 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:15:31.549429 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:15:31.550145 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:16:31.554933 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:16:31.555612 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:17:31.550735 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:17:31.551408 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:18:31.555102 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:18:31.555751 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:19:31.559365 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:19:31.560022 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:20:31.564250 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:20:31.564929 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:21:31.568439 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:21:31.569136 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:22:31.573707 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:22:31.574276 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jan 29 22:23:31.579400 ncdv-master-1 cinder-backup[134197]: DEBUG oslo_service.periodic_task [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=134197) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jan 29 22:23:31.580080 ncdv-master-1 cinder-backup[134197]: DEBUG cinder.manager [None req-6b222709-f956-4f74-8bd0-2b336c7ae2e9 None None] Notifying Schedulers of capabilities ... {{(pid=134197) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}