-- Logs begin at Sat 2022-05-14 20:01:12 UTC, end at Sat 2022-05-14 22:44:14 UTC. -- May 14 21:01:52.370017 ncdv-master-1 systemd[1]: Started Devstack devstack@c-vol.service. May 14 21:01:55.885636 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=123700) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 14 21:01:55.886232 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=123700) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 14 21:01:56.284309 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_db.sqlalchemy.engines [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123700) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 14 21:01:56.503926 ncdv-master-1 cinder-volume[123700]: INFO cinder.rpc [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. May 14 21:01:56.508475 ncdv-master-1 cinder-volume[123700]: INFO cinder.rpc [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. May 14 21:01:56.516883 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_db.sqlalchemy.engines [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123700) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 14 21:01:56.680349 ncdv-master-1 cinder-volume[123700]: INFO cinder.volume.manager [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Determined volume DB was empty at startup. May 14 21:01:56.685608 ncdv-master-1 cinder-volume[123700]: DEBUG cinder.volume.manager [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Cinder Volume DB check: vol_db_empty=True {{(pid=123700) __init__ /opt/stack/cinder/cinder/volume/manager.py:286}} May 14 21:01:56.690112 ncdv-master-1 cinder-volume[123700]: INFO cinder.volume.manager [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Service not found for updating active_backend_id, assuming default for driver init. May 14 21:01:56.690890 ncdv-master-1 cinder-volume[123700]: WARNING stevedore.named [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager May 14 21:01:56.691316 ncdv-master-1 cinder-volume[123700]: WARNING castellan.key_manager [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' May 14 21:01:56.692456 ncdv-master-1 cinder-volume[123700]: WARNING cinder.keymgr.conf_key_mgr [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] This key manager is insecure and is not recommended for production deployments: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' May 14 21:01:56.708908 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_db.sqlalchemy.engines [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123700) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 14 21:01:56.718275 ncdv-master-1 cinder-volume[123700]: INFO cinder.rpc [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. May 14 21:01:56.718275 ncdv-master-1 cinder-volume[123700]: INFO cinder.rpc [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Automatically selected cinder-volume RPC version 3.18 as minimum service version. May 14 21:01:56.719632 ncdv-master-1 cinder-volume[123700]: WARNING stevedore.named [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager May 14 21:01:56.719632 ncdv-master-1 cinder-volume[123700]: WARNING castellan.key_manager [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' May 14 21:01:56.728499 ncdv-master-1 cinder-volume[123700]: INFO cinder.volume.manager [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Image-volume cache enabled for host ncdv-master-1@ceph. May 14 21:01:56.728499 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_db.sqlalchemy.engines [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123700) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 14 21:01:56.792524 ncdv-master-1 cinder-volume[123700]: INFO oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Starting 1 workers May 14 21:01:56.799837 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Started child 123870 {{(pid=123700) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} May 14 21:01:56.801199 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Full set of CONF: {{(pid=123700) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} May 14 21:01:56.801779 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ******************************************************************************** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 14 21:01:56.802257 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] Configuration options gathered from: {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 14 21:01:56.802811 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 14 21:01:56.803607 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 14 21:01:56.804061 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ================================================================================ {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 14 21:01:56.804961 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] allow_availability_zone_fallback = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.805453 ncdv-master-1 cinder-volume[123870]: INFO cinder.service [-] Starting cinder-volume node (version 20.1.0) May 14 21:01:56.806476 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] allow_compression_on_image_upload = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.807215 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] allowed_direct_url_schemes = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.807782 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] api_rate_limit = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.808401 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] auth_strategy = keystone {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.809045 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] az_cache_duration = 3600 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.809631 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_availability_zone = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.810262 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_host = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.810785 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.811380 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_stats_polling_interval = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.812055 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backup_api_class = cinder.backup.api.API {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.812612 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.813229 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backup_name_template = backup-%s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.813831 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backup_use_same_host = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.814430 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backup_use_temp_snapshot = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.814998 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] chap_password = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.815692 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] chap_username = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.816463 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.817097 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_internal_tenant_project_id = 0911a024287e430ea6ee015389f0c440 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.817664 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_internal_tenant_user_id = 198271d1c0d44cb58bace41a65f4b94c {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.818204 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cloned_volume_same_az = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.820033 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cluster = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.820739 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] compression_format = gzip {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.821366 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] compute_api_class = cinder.compute.nova.API {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.822030 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] config_dir = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.822502 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.823038 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] config_source = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.823593 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.824151 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] control_exchange = openstack {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.824749 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] debug = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.825328 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] default_availability_zone = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.826015 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] default_group_type = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.827197 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.827820 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] default_volume_type = ceph {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.828354 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] driver_client_cert = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.828906 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] driver_client_cert_key = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.829464 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] driver_data_namespace = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.830011 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] driver_ssl_cert_path = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.830617 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] driver_ssl_cert_verify = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.831147 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] driver_use_ssl = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.831777 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] enable_force_upload = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.832253 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] enable_new_services = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.832779 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] enable_unsupported_driver = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.833418 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] enabled_backends = ['ceph'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.834030 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] enforce_multipath_for_image_xfer = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.834626 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] filter_function = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.835172 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_api_insecure = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.835904 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_api_servers = ['http://10.222.0.52/image'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.836421 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_api_ssl_compression = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.837190 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_ca_certificates_file = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.837789 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_catalog_info = image:glance:publicURL {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.838398 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.839073 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.839728 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.840526 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_num_retries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.844746 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] glance_request_timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.845364 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] goodness_function = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.846107 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] graceful_shutdown_timeout = 5 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.846693 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] group_api_class = cinder.group.api.API {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.847371 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] host = ncdv-master-1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.847940 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iet_conf = /etc/iet/ietd.conf {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.848517 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_compress_on_upload = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.849071 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_conversion_address_space_limit = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.849652 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_conversion_cpu_limit = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.850365 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.850909 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_upload_use_cinder_backend = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.851442 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_upload_use_internal_tenant = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.851797 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Starting volume driver RBDDriver (1.2.0) May 14 21:01:56.852164 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_volume_cache_enabled = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.852684 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_volume_cache_max_count = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.853214 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] image_volume_cache_max_size_gb = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.853826 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] init_host_max_objects_retrieval = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.854205 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:56.854525 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] instance_format = [instance: %(uuid)s] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.855067 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.855649 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iscsi_iotype = fileio {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.856314 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iscsi_secondary_ip_addresses = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.856842 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iscsi_target_flags = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.857622 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iscsi_write_cache = on {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.858206 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iser_helper = tgtadm {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.859571 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iser_ip_address = 10.222.0.52 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.868308 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iser_port = 3260 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.868973 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.869543 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_config_append = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.870109 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.870623 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_dir = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.871204 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_file = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.871694 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_options = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.872229 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_rotate_interval = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.872767 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_rotate_interval_type = days {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.873309 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] log_rotation_type = none {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.873847 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.874414 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.875091 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.875634 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.876442 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.877012 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] max_age = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.877614 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] max_logfile_count = 30 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.878236 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] max_logfile_size_mb = 200 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.878794 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] max_over_subscription_ratio = 20.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.879395 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] message_reap_interval = 86400 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.879934 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] message_ttl = 2592000 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.880467 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] migration_create_volume_timeout_secs = 300 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.880971 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] monkey_patch = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.881562 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] monkey_patch_modules = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.882026 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] my_ip = 10.222.0.52 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.882545 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] no_snapshot_gb_quota = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.883041 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] num_iser_scan_tries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.883592 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] num_shell_tries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.884098 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] num_volume_device_scan_tries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.884617 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] nvmet_ns_id = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.885131 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] nvmet_port_id = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.885678 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_max_limit = 1000 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.886808 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_volume_ext_list = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.887453 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.888232 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_volume_listen = 0.0.0.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.888782 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_volume_listen_port = 8776 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.890142 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_volume_use_ssl = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.890766 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] osapi_volume_workers = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.891327 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] per_volume_size_limit = -1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.891862 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] periodic_fuzzy_delay = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.892385 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] periodic_interval = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.892955 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] project_id_regex = [0-9a-f\-]+ {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.893468 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] public_endpoint = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.893978 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] publish_errors = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.894490 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_backup_gigabytes = 1000 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.895010 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_backups = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.895656 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_consistencygroups = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.896183 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.897130 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_gigabytes = 1000 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.898224 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_groups = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.898861 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_snapshots = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.899408 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] quota_volumes = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.899964 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] rate_limit_burst = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.900489 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] rate_limit_except_level = CRITICAL {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.901017 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] rate_limit_interval = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.901532 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] reinit_driver_count = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.902048 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] replication_device = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.902564 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] report_discard_supported = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.903093 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] report_interval = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.903738 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] reservation_clean_interval = 86400 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.904195 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] reservation_expire = 86400 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.904707 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] reserved_percentage = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.905894 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.906533 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.907237 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] rpc_response_timeout = 120 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.907849 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] run_external_periodic_tasks = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.908506 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.909150 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] scst_target_driver = iscsi {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.909650 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] scst_target_iqn_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.910092 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_down_time = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.910626 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] snapshot_name_template = snapshot-%s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.911158 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] snapshot_same_host = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.911737 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] split_loggers = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.912293 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] state_path = /opt/stack/data/cinder {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.912809 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] storage_availability_zone = nova {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.913355 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] storage_protocol = iSCSI {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.913888 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] syslog_log_facility = LOG_USER {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.918054 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] target_helper = lioadm {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.918054 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] target_ip_address = 10.222.0.52 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.918054 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] target_port = 3260 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.918054 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.918054 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] target_protocol = iscsi {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.918736 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] trace_flags = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.919207 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] transfer_api_class = cinder.transfer.api.API {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.919760 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] transport_url = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.921432 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] until_refresh = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.922073 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_chap_auth = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.922553 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_default_quota_class = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.923052 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_eventlog = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.931454 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_journal = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.932245 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_json = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.932772 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_multipath_for_image_xfer = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.933303 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_stderr = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.933842 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] use_syslog = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.934238 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:56.934569 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] verify_glance_signatures = enabled {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.935089 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_api_class = cinder.volume.api.API {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.935678 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_backend_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.936224 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_clear = zero {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.937572 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_clear_ionice = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.938113 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_clear_size = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.938659 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.939181 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_copy_bps_limit = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.939893 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_dd_blocksize = 1M {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.941472 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.942007 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_name_template = volume-%s {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.942518 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_service_inithost_offload = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.943046 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volume_usage_audit_period = month {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.943778 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.944366 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] watch_log_file = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.944870 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] zoning_mode = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 14 21:01:56.946388 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.947061 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.947737 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_concurrency.disable_process_locking = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.948451 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.949081 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.949669 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.950191 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_metrics.metrics_process_name = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.951780 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.952936 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.953709 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.admin_password = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.954296 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.admin_tenant_name = admin {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.954826 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.admin_token = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.956089 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.admin_user = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.956648 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_admin_prefix = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.957460 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.958047 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_port = 35357 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.958590 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_protocol = https {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.959169 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_section = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.959813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_type = password {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.960297 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:56.960700 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_uri = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.961985 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.auth_version = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.962516 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.cache = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.963090 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.963640 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.964179 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.delay_auth_decision = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.964790 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.965299 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.http_connect_timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.965809 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.966315 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.identity_uri = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.966823 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.include_service_catalog = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.967366 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.insecure = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.967990 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.interface = public {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.968679 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.969191 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.969730 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.970245 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.970916 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.971680 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.972194 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_secret_key = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.972573 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_security_strategy = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.972936 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.973351 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.973710 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.region_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.974087 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.974441 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.service_token_roles_required = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.974788 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.service_type = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.975138 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.token_cache_time = 300 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.975878 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] keystone_authtoken.www_authenticate_uri = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.976891 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.backend = dogpile.cache.null {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.977548 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.backend_argument = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.977918 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.config_prefix = cache.oslo {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.978544 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.dead_timeout = 60.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.978921 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.debug_cache_backend = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.979280 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.enable_retry_client = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.979665 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.enable_socket_keepalive = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.980039 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.enabled = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.980410 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.expiration_time = 600 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.980760 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.hashclient_retry_attempts = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.981388 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.hashclient_retry_delay = 1.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.982370 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_dead_retry = 300 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.982760 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.983119 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.983568 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_pool_maxsize = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.983930 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_pool_unused_timeout = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.984334 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_servers = ['localhost:11211'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.984721 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.memcache_socket_timeout = 1.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.985170 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.proxies = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.985709 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.retry_attempts = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.986286 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.retry_delay = 0.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.986798 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.socket_keepalive_count = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.987310 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.socket_keepalive_idle = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.987770 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.socket_keepalive_interval = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.988129 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.tls_allowed_ciphers = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.988480 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.tls_cafile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.988987 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.tls_certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.989486 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.tls_enabled = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.990197 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cache.tls_keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.990930 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] coordination.backend_url = etcd3+http://10.222.0.52:2379 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.991976 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.backend = sqlalchemy {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.992548 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.connection = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.992949 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.connection_debug = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.993336 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.connection_parameters = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.993712 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.connection_recycle_time = 3600 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.994072 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.connection_trace = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.994444 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.db_inc_retry_interval = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.994817 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.db_max_retries = 20 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.995166 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.db_max_retry_interval = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.995704 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.db_retry_interval = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.996121 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.max_overflow = 50 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.996686 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.max_pool_size = 5 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.997057 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.max_retries = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.997415 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.mysql_enable_ndb = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.997784 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.mysql_sql_mode = TRADITIONAL {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.998145 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.pool_timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.998537 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.retry_interval = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.999425 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.slave_connection = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:56.999836 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.sqlite_synchronous = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.000290 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] database.use_db_reconnect = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.000729 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.enforce_new_defaults = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.001265 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.driver [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Driver hasn't implemented _init_vendor_properties() May 14 21:01:57.001459 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.enforce_scope = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.001818 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.driver [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:5edbe80d-a532-477d-85bc-c68f9b0447a0:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=123870) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}} May 14 21:01:57.002090 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.policy_default_rule = default {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.002591 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.002988 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.policy_file = policy.yaml {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.003381 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.003755 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.004106 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.004469 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.004817 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.005299 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.connection_string = messaging:// {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.005660 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.enabled = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.006022 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.es_doc_type = notification {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.006392 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.es_scroll_size = 10000 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.006749 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.es_scroll_time = 2m {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.007096 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.filter_error_trace = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.007457 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.hmac_keys = SECRET_KEY {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.007831 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.sentinel_service_name = mymaster {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.008262 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.socket_timeout = 0.1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.008639 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] profiler.trace_sqlalchemy = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.009051 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] privsep_osbrick.capabilities = [21] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.009425 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] privsep_osbrick.group = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.009766 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] privsep_osbrick.helper_command = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.010124 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.010572 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] privsep_osbrick.thread_pool_size = 4 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.010947 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] privsep_osbrick.user = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.011360 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.011761 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_sys_admin.group = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.012324 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_sys_admin.helper_command = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.012815 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.013164 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.013507 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] cinder_sys_admin.user = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.013898 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.auth_section = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.014250 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.auth_type = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.015283 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.cafile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.015746 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.016123 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.collect_timing = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.016471 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.insecure = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.016812 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.017181 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.send_service_user_token = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.017538 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.split_loggers = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.017886 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] service_user.timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.018353 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.backend_availability_zone = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.018713 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.backend_native_threads_pool_size = 20 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.019252 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.backup_use_temp_snapshot = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.019669 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.chap_password = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.020034 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.chap_username = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.020422 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.020786 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.021160 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.021503 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.driver_client_cert = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.021842 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.driver_client_cert_key = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.022184 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.driver_data_namespace = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.022740 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.driver_ssl_cert_path = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.023512 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.023512 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.driver_use_ssl = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.025583 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.enable_deferred_deletion = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.025583 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.enable_unsupported_driver = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.025583 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.025583 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.extra_capabilities = {} {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.025583 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.filter_function = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.025583 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.goodness_function = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.026118 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.026243 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.026596 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.026936 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.image_volume_cache_enabled = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.027048 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_db.sqlalchemy.engines [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123870) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 14 21:01:57.027417 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.027647 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.028006 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iscsi_iotype = fileio {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.028366 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.028725 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iscsi_target_flags = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.029083 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iscsi_write_cache = on {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.029446 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iser_helper = tgtadm {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.029989 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iser_ip_address = 10.222.0.52 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.030377 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iser_port = 3260 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.030749 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.031114 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.031470 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.031837 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.num_shell_tries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.032181 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.032704 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.nvmet_ns_id = 10 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.033058 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.nvmet_port_id = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.033408 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rados_connect_timeout = -1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.033750 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rados_connection_interval = 5 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.034102 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rados_connection_retries = 3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.034462 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_ceph_conf = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.034823 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_cluster_name = ceph {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.035198 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041115 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041115 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041115 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_pool = rbd {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041115 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_secret_uuid = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041115 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041115 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.rbd_user = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.replication_connect_timeout = 5 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.replication_device = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.report_discard_supported = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.reserved_percentage = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.041813 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.scst_target_driver = iscsi {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.042905 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.scst_target_iqn_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.042905 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.storage_protocol = iSCSI {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.042905 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.suppress_requests_ssl_warnings = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.042905 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.target_helper = tgtadm {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.042905 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.target_ip_address = 10.222.0.52 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.042905 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.target_port = 3260 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.043415 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.043415 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.target_protocol = iscsi {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.043415 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.trace_flags = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.045071 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.unique_fqdn_network = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.045436 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.use_chap_auth = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.045830 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.046182 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_backend_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.046631 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_clear = zero {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.046901 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_clear_ionice = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.047267 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_clear_size = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.047651 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.048022 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.048392 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.048776 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.049267 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.049657 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_reports.file_event_handler = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.050029 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_reports.file_event_handler_interval = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.050458 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_reports.log_dir = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.051064 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.backend_availability_zone = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.051491 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.backend_host = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.051799 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.backend_native_threads_pool_size = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.052157 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.backup_use_temp_snapshot = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.052519 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.chap_password = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.053028 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.chap_username = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.053384 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.chiscsi_conf = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.053733 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.deferred_deletion_delay = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.054076 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.deferred_deletion_purge_interval = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.055223 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.driver_client_cert = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.055603 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.driver_client_cert_key = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.056010 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.driver_data_namespace = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.056358 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.driver_ssl_cert_path = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.056715 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.driver_ssl_cert_verify = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.057061 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.driver_use_ssl = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.057414 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.enable_deferred_deletion = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.057762 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.enable_unsupported_driver = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.058107 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.enforce_multipath_for_image_xfer = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.058451 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.extra_capabilities = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.058793 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.filter_function = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.059133 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.goodness_function = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.059593 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iet_conf = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.059879 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.image_upload_use_cinder_backend = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.060223 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.image_upload_use_internal_tenant = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.060587 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.image_volume_cache_enabled = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.060930 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.image_volume_cache_max_count = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.061273 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.image_volume_cache_max_size_gb = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.062527 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iscsi_iotype = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.063270 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iscsi_secondary_ip_addresses = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.063270 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iscsi_target_flags = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.063547 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iscsi_write_cache = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.063900 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iser_helper = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.064266 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iser_ip_address = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.065346 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iser_port = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.065701 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.iser_target_prefix = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.066051 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.max_over_subscription_ratio = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.066401 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.num_iser_scan_tries = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.066746 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.num_shell_tries = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.067094 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.num_volume_device_scan_tries = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.067438 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.nvmet_ns_id = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.067814 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.nvmet_port_id = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.068221 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rados_connect_timeout = -1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.068569 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rados_connection_interval = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.068915 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rados_connection_retries = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.069273 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_ceph_conf = /etc/ceph/ceph.conf {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.069614 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_cluster_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.069991 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_exclusive_cinder_pool = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.070381 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_flatten_volume_from_snapshot = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.070794 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_max_clone_depth = 5 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.071163 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_pool = volumes {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.071580 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_secret_uuid = b3399bd5-b147-45ea-b0ac-77d4339d7906 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.071945 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_store_chunk_size = 4 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.072303 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.rbd_user = cinder {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.072652 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.replication_connect_timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.073111 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.replication_device = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.073525 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.report_discard_supported = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.073872 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.report_dynamic_total_capacity = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.074222 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.reserved_percentage = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.074962 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.scst_target_driver = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.075768 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.scst_target_iqn_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.076119 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.storage_protocol = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.076467 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.suppress_requests_ssl_warnings = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.076811 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.target_helper = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.077743 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.target_ip_address = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.078122 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.target_port = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.078498 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.target_prefix = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.078844 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.target_protocol = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.080896 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.trace_flags = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.080896 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.unique_fqdn_network = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.080896 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.use_chap_auth = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.080896 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.use_multipath_for_image_xfer = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.080896 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_backend_name = ceph {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.080896 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_clear = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.081426 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_clear_ionice = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.081573 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_clear_size = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.081926 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_copy_blkio_cgroup_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.082275 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_copy_bps_limit = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.082650 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_dd_blocksize = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.083744 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volume_driver = cinder.volume.drivers.rbd.RBDDriver {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.084099 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ceph.volumes_dir = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.084770 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.085738 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.086117 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.086477 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.086902 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.087253 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.087650 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.088067 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.088420 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.088762 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.089247 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.089627 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.090096 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.090455 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.090801 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.091163 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.091687 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.092019 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.092403 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.092930 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.093478 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.093960 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.094366 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.ssl = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.095099 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.095888 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.096243 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.096537 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.db.sqlalchemy.api [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Building query based on filter {{(pid=123870) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3550}} May 14 21:01:57.096675 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.096995 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_rabbit.ssl_version = {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.097333 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_notifications.driver = [] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.097700 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_notifications.retry = -1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.098135 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.098486 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] oslo_messaging_notifications.transport_url = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.098838 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.099134 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] key_manager.fixed_key = **** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.099669 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.100119 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.barbican_api_version = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.100497 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.barbican_endpoint = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.100880 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.barbican_endpoint_type = public {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.101223 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.barbican_region_name = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.101577 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.cafile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.101923 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.102288 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.collect_timing = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.102635 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.insecure = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.102980 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.103342 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.number_of_retries = 60 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.103719 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.retry_delay = 1 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.104183 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.send_service_user_token = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.104542 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.split_loggers = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.105260 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.106070 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.verify_ssl = True {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.106428 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican.verify_ssl_path = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.106808 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.auth_section = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.107161 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.auth_type = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.107650 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.cafile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.108000 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.108894 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.collect_timing = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.109450 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.insecure = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.109792 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.110146 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.split_loggers = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.110489 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] barbican_service_user.timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.110876 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.approle_role_id = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.111219 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.approle_secret_id = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.111585 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.cafile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.111931 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.certfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.112287 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.collect_timing = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.112625 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.insecure = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.112963 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.keyfile = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.113325 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.kv_mountpoint = secret {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.113678 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.kv_version = 2 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.114153 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.namespace = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.114585 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.root_token_id = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.114929 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.split_loggers = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.115938 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.ssl_ca_crt_file = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.116399 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.timeout = None {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.116755 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.use_ssl = False {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.117122 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 14 21:01:57.117409 ncdv-master-1 cinder-volume[123700]: DEBUG oslo_service.service [None req-6a433f03-6006-4e19-a29b-acc79a0d1b9d None None] ******************************************************************************** {{(pid=123700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 14 21:01:57.126537 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Configuration image_conversion_dir is None or the path doesn't exist. {{(pid=123870) cleanup_temporary_file /opt/stack/cinder/cinder/image/image_utils.py:957}} May 14 21:01:57.135956 ncdv-master-1 cinder-volume[123870]: INFO cinder.keymgr.migration [None req-ff7747e5-ee42-405e-b7bd-f600f612721d None None] Not migrating encryption keys because the ConfKeyManager is still in use. May 14 21:01:57.140239 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:57.171289 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:57.210389 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:01:57.235622 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Driver initialization completed successfully. May 14 21:01:57.236595 ncdv-master-1 cinder-volume[123870]: INFO cinder.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Initiating service 3 cleanup May 14 21:01:57.245881 ncdv-master-1 cinder-volume[123870]: INFO cinder.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Service 3 cleanup completed. May 14 21:01:57.246464 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.service [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Creating RPC server for service cinder-volume {{(pid=123870) start /opt/stack/cinder/cinder/service.py:231}} May 14 21:01:57.254645 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.service [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=123870) start /opt/stack/cinder/cinder/service.py:237}} May 14 21:01:57.299889 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Initializing RPC dependent components of volume driver RBDDriver (1.2.0) May 14 21:01:57.301203 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:57.335031 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:01:57.459188 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f6ff4e67-8413-4a90-8ddd-5aa6d2dc7b1c None None] Driver post RPC initialization completed successfully. May 14 21:03:02.462283 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:03:02.463093 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:03:02.486161 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:03:02.513527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:04:02.459590 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:04:02.460267 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:04:02.500257 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:04:02.523973 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:05:02.459199 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:05:02.460396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:05:02.500025 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:05:02.523059 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:06:02.459585 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:06:02.460285 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:06:02.482231 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:06:02.505355 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:07:02.459987 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:07:02.460669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:07:02.482680 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:07:02.504838 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:08:02.459942 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:08:02.460684 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:08:02.482515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:08:02.504342 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:09:02.459665 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:09:02.460396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:09:02.482097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:09:02.503813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:11:02.460949 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:11:02.461612 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:11:02.483270 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:11:02.506486 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:12:02.462387 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:12:02.463874 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:12:02.531313 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:12:02.597026 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:13:02.463422 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:13:02.464134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:13:02.493032 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:13:02.519951 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:14:02.465960 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:14:02.466716 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:14:02.545323 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:14:02.695354 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:15:02.461347 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:15:02.462029 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:15:02.534437 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:15:02.638225 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:15:54.513122 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:15:54.583277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Flow 'volume_create_manager' (d8643135-305b-4699-8696-13ea07a63039) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:15:54.601931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c2d50dc5-4df8-4b10-839c-c80c0feb3482) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:15:54.671598 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c2d50dc5-4df8-4b10-839c-c80c0feb3482) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:15:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveMigrationTest-volume-2067339704',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e0863bd3-bff3-4125-b857-a7cf7b43cf86,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='763cf73206604d8abbe1542afe44fa95',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:15:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:15:54Z,use_quota=True,user_id='038739f45c3d4ab09457d6eed20b41d4',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:15:54.676796 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (82afbde4-f528-407a-995b-5a974b8de06b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:15:54.682510 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (82afbde4-f528-407a-995b-5a974b8de06b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:15:54.687310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8fc4760c-8835-40e9-ac1d-9bc0c9839f98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:15:54.691325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8fc4760c-8835-40e9-ac1d-9bc0c9839f98) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'volume_name': 'volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:15:54.706359 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (85217121-1d83-4259-8811-f4cec4a2d947) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:15:54.713832 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (85217121-1d83-4259-8811-f4cec4a2d947) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:15:54.721810 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1a0ae972-190a-4102-8940-2fcbcb115f0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:15:54.724429 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Volume e0863bd3-bff3-4125-b857-a7cf7b43cf86: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'volume_size': 1} May 14 21:15:54.725280 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] creating volume 'volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:15:54.738479 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:15:55.709699 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:15:55.710835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:15:55.715211 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1a0ae972-190a-4102-8940-2fcbcb115f0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:15:55.721520 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7e907459-4776-470d-b04d-c19a8b4fd5d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:15:55.820985 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Volume volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86 (e0863bd3-bff3-4125-b857-a7cf7b43cf86): created successfully May 14 21:15:55.825916 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7e907459-4776-470d-b04d-c19a8b4fd5d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:15:55.831717 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Flow 'volume_create_manager' (d8643135-305b-4699-8696-13ea07a63039) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:15:55.975355 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0e92e33e-0c44-4e59-b9e9-dc1fe9977152 tempest-LiveMigrationTest-772526 None] Created volume successfully. May 14 21:16:00.033928 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-610e705e-aeff-4fab-ac56-ebe50d32f837 req-1f120cab-f58e-4b39-ac09-1fbb7aebd46d tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:16:01.078272 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-610e705e-aeff-4fab-ac56-ebe50d32f837 req-1f120cab-f58e-4b39-ac09-1fbb7aebd46d tempest-LiveMigrationTest-772526 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.030s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:16:01.078272 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-610e705e-aeff-4fab-ac56-ebe50d32f837 req-1f120cab-f58e-4b39-ac09-1fbb7aebd46d tempest-LiveMigrationTest-772526 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:16:01.195472 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-610e705e-aeff-4fab-ac56-ebe50d32f837 req-1f120cab-f58e-4b39-ac09-1fbb7aebd46d tempest-LiveMigrationTest-772526 None] Connection info returned from driver {'name': 'volumes/volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '33505cfa-e730-4b2f-ab26-eb2ee49ce933'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:16:01.362330 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-610e705e-aeff-4fab-ac56-ebe50d32f837 req-1f120cab-f58e-4b39-ac09-1fbb7aebd46d tempest-LiveMigrationTest-772526 None] attachment_update completed successfully. May 14 21:16:02.462117 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:16:02.465818 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:16:02.517471 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:16:02.582726 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:16:18.483374 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-fec0471c-df68-4ab2-8087-6654710c745c tempest-LiveMigrationTest-151813528 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:16:19.386286 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-fec0471c-df68-4ab2-8087-6654710c745c tempest-LiveMigrationTest-151813528 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.902s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:16:19.387022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-fec0471c-df68-4ab2-8087-6654710c745c tempest-LiveMigrationTest-151813528 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:16:19.697644 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-fec0471c-df68-4ab2-8087-6654710c745c tempest-LiveMigrationTest-151813528 None] Connection info returned from driver {'name': 'volumes/volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e0863bd3-bff3-4125-b857-a7cf7b43cf86', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd0472ae6-ee1c-49d8-935c-f152b6d3d7ec'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:16:19.901255 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-fec0471c-df68-4ab2-8087-6654710c745c tempest-LiveMigrationTest-151813528 None] attachment_update completed successfully. May 14 21:16:34.855554 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-38d4ad79-eaab-4b2c-b552-55ef292f58e4 tempest-LiveMigrationTest-151813528 None] Terminate volume connection completed successfully. May 14 21:16:34.856403 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-09c9a0ec-ea97-4263-84d2-50374d0a77fc req-38d4ad79-eaab-4b2c-b552-55ef292f58e4 tempest-LiveMigrationTest-151813528 None] Deleting attachment 33505cfa-e730-4b2f-ab26-eb2ee49ce933. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:16:44.535405 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-a487d452-5b78-49a5-957b-879a54d739f2 req-d5ff0c32-0ba8-40d7-865b-e3637c286052 tempest-LiveMigrationTest-772526 None] Terminate volume connection completed successfully. May 14 21:16:44.536071 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-a487d452-5b78-49a5-957b-879a54d739f2 req-d5ff0c32-0ba8-40d7-865b-e3637c286052 tempest-LiveMigrationTest-772526 None] Deleting attachment d0472ae6-ee1c-49d8-935c-f152b6d3d7ec. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:16:46.881498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:16:46.948880 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Flow 'volume_create_manager' (3ab48f9e-cbd9-4d68-b60c-e75c6578a9ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:16:46.958019 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6a670f2f-1e4b-42b0-9445-0a623ef82739) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:16:46.976283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6a670f2f-1e4b-42b0-9445-0a623ef82739) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:16:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1108329103',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a73964bc-4e69-4eef-b351-fcd6ec89ade1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='763cf73206604d8abbe1542afe44fa95',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:16:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:16:47Z,use_quota=True,user_id='038739f45c3d4ab09457d6eed20b41d4',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:16:46.985642 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (37a77753-4f66-4da0-afec-ed21f20281e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:16:46.990414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (37a77753-4f66-4da0-afec-ed21f20281e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:16:46.996394 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e5c721ce-d213-4b18-827b-5c7d3cb7ef42) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:16:47.115833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e5c721ce-d213-4b18-827b-5c7d3cb7ef42) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'volume_name': 'volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:16:47.122480 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ff3d12b7-c25f-4e93-8f91-4628d11e5fc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:16:47.127670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ff3d12b7-c25f-4e93-8f91-4628d11e5fc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:16:47.131043 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b598e34-1dba-48c5-b859-7c4673eee718) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:16:47.132226 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume a73964bc-4e69-4eef-b351-fcd6ec89ade1: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:16:47.132965 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Cloning a73964bc-4e69-4eef-b351-fcd6ec89ade1 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:16:47.147199 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:16:47.154461 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Preparing cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:879}} May 14 21:16:47.155833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:17:02.467616 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:17:02.468377 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:17:02.575372 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:17:02.635978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:17:10.999640 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Image fetch details: dest /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph, size 616.00 MB, duration 23.84 sec {{(pid=123870) fetch /opt/stack/cinder/cinder/image/image_utils.py:582}} May 14 21:17:11.000329 ncdv-master-1 cinder-volume[123870]: INFO cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Image download 616.00 MB at 25.84 MB/s May 14 21:17:11.087521 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:11.206208 ncdv-master-1 sudo[135539]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph May 14 21:17:11.207109 ncdv-master-1 sudo[135539]: pam_unix(sudo:session): session opened for user root by (uid=0) May 14 21:17:13.325408 ncdv-master-1 sudo[135539]: pam_unix(sudo:session): session closed for user root May 14 21:17:13.327636 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph" returned: 0 in 2.240s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:13.331613 ncdv-master-1 cinder-volume[123870]: WARNING py.warnings [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 14 21:17:13.331613 ncdv-master-1 cinder-volume[123870]: debtcollector.deprecate( May 14 21:17:13.331613 ncdv-master-1 cinder-volume[123870]: May 14 21:17:13.334635 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Temporary image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e is fetched for user 038739f45c3d4ab09457d6eed20b41d4. {{(pid=123870) fetch /opt/stack/cinder/cinder/image/image_utils.py:1090}} May 14 21:17:13.497389 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:13.698459 ncdv-master-1 sudo[135590]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph May 14 21:17:13.699930 ncdv-master-1 sudo[135590]: pam_unix(sudo:session): session opened for user root by (uid=0) May 14 21:17:15.442336 ncdv-master-1 sudo[135590]: pam_unix(sudo:session): session closed for user root May 14 21:17:15.446328 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph" returned: 0 in 1.949s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:15.447322 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] creating volume 'volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:17:15.449422 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:17:16.112372 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:17:16.112816 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:17:16.329107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Attempting download of b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e ((None, None)) to volume a73964bc-4e69-4eef-b351-fcd6ec89ade1. {{(pid=123870) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1179}} May 14 21:17:16.403315 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_sl6sci08 {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:16.638874 ncdv-master-1 sudo[135684]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_sl6sci08 May 14 21:17:16.639689 ncdv-master-1 sudo[135684]: pam_unix(sudo:session): session opened for user root by (uid=0) May 14 21:17:18.829017 ncdv-master-1 sudo[135684]: pam_unix(sudo:session): session closed for user root May 14 21:17:18.831935 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_sl6sci08" returned: 0 in 2.428s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:18.833826 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:19.130639 ncdv-master-1 sudo[135725]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph May 14 21:17:19.132925 ncdv-master-1 sudo[135725]: pam_unix(sudo:session): session opened for user root by (uid=0) May 14 21:17:20.224418 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:17:20.284079 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Flow 'volume_create_manager' (26f9fbc0-f915-4194-96bb-9b4f80bef200) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:17:20.301149 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (48252086-6897-4178-b4dc-f3fb9cb7e65b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:20.335178 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (48252086-6897-4178-b4dc-f3fb9cb7e65b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:17:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-357194704',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=022c11da-58a1-4855-977c-de51c3870d3f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ef1f3155d52e42bb9c54f43a8360a74e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:17:20Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:17:20Z,use_quota=True,user_id='4e87a5a5713748b6a0d73984a279009b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:20.343971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0a58705-7490-4322-8484-4ad9cbbef5db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:20.348773 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0a58705-7490-4322-8484-4ad9cbbef5db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:20.355110 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1fd003bd-601c-4828-a6f2-7a1e764d5dcf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:20.480123 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1fd003bd-601c-4828-a6f2-7a1e764d5dcf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '022c11da-58a1-4855-977c-de51c3870d3f', 'volume_name': 'volume-022c11da-58a1-4855-977c-de51c3870d3f', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:20.488325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e3e96cac-5006-4ed4-8bd8-a6b9460bdd6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:20.505729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e3e96cac-5006-4ed4-8bd8-a6b9460bdd6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:20.528850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6c381059-779e-41b3-932c-06d8fe64a40b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:20.534043 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Volume 022c11da-58a1-4855-977c-de51c3870d3f: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-022c11da-58a1-4855-977c-de51c3870d3f', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:17:20.534737 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Cloning 022c11da-58a1-4855-977c-de51c3870d3f from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:17:21.456430 ncdv-master-1 sudo[135725]: pam_unix(sudo:session): session closed for user root May 14 21:17:21.460822 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph" returned: 0 in 2.626s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:21.462114 ncdv-master-1 cinder-volume[123870]: WARNING py.warnings [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 14 21:17:21.462114 ncdv-master-1 cinder-volume[123870]: debtcollector.deprecate( May 14 21:17:21.462114 ncdv-master-1 cinder-volume[123870]: May 14 21:17:21.463406 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e was vhdx, converting to raw {{(pid=123870) fetch_to_volume_format /opt/stack/cinder/cinder/image/image_utils.py:795}} May 14 21:17:21.464667 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpyt28ab82 {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:21.525918 ncdv-master-1 sudo[135816]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpyt28ab82 May 14 21:17:21.536886 ncdv-master-1 sudo[135816]: pam_unix(sudo:session): session opened for user root by (uid=0) May 14 21:17:24.675607 ncdv-master-1 sudo[135816]: pam_unix(sudo:session): session closed for user root May 14 21:17:24.678943 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpyt28ab82" returned: 0 in 3.213s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:24.680049 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:24.797548 ncdv-master-1 sudo[135880]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph May 14 21:17:24.799113 ncdv-master-1 sudo[135880]: pam_unix(sudo:session): session opened for user root by (uid=0) May 14 21:17:24.991905 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:17:25.019935 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Flow 'volume_create_manager' (2cce304c-a03e-4cb7-92f5-b60f21f950dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:17:25.021213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f3083157-798a-4798-8ecf-8c271e56e83d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:25.047784 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f3083157-798a-4798-8ecf-8c271e56e83d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:17:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveAutoBlockMigrationV225Test-volume-642082702',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2a72aa90-a850-47be-bc0c-572dbc34980a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='74bb7b020fae4bcaa3a440acc60c7a1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:17:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:17:24Z,use_quota=True,user_id='cfcb989a491440b3bf74ddc055ee14d3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:25.064867 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4a4a85c2-a879-4015-8191-1b0026d9bf89) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:25.073280 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4a4a85c2-a879-4015-8191-1b0026d9bf89) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:25.083925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d3069a7-b138-49c9-bea6-88edf6507e43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:25.097239 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d3069a7-b138-49c9-bea6-88edf6507e43) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2a72aa90-a850-47be-bc0c-572dbc34980a', 'volume_name': 'volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:25.107923 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (17e83f98-435f-4b55-abde-a3cac73c6da8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:25.122686 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (17e83f98-435f-4b55-abde-a3cac73c6da8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:25.132465 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cd298f17-85ee-4547-9d89-3b3534d85a82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:25.136208 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume 2a72aa90-a850-47be-bc0c-572dbc34980a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'volume_size': 1} May 14 21:17:25.137013 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] creating volume 'volume-2a72aa90-a850-47be-bc0c-572dbc34980a' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:17:25.138508 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:17:25.713945 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:17:25.714641 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:17:25.720873 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cd298f17-85ee-4547-9d89-3b3534d85a82) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:25.723310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cc8d4325-04b0-45b2-9a9c-65dafb83a51c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:17:25.781622 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume volume-2a72aa90-a850-47be-bc0c-572dbc34980a (2a72aa90-a850-47be-bc0c-572dbc34980a): created successfully May 14 21:17:25.785998 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cc8d4325-04b0-45b2-9a9c-65dafb83a51c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:17:25.794770 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Flow 'volume_create_manager' (2cce304c-a03e-4cb7-92f5-b60f21f950dc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:17:25.953513 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f1e46b8b-5766-4a21-8fc1-5790f0ea5ca1 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Created volume successfully. May 14 21:17:26.261367 ncdv-master-1 sudo[135880]: pam_unix(sudo:session): session closed for user root May 14 21:17:26.263863 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph" returned: 0 in 1.584s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:26.269354 ncdv-master-1 cinder-volume[123870]: WARNING py.warnings [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 14 21:17:26.269354 ncdv-master-1 cinder-volume[123870]: debtcollector.deprecate( May 14 21:17:26.269354 ncdv-master-1 cinder-volume[123870]: May 14 21:17:26.269354 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Image conversion details: src /opt/stack/data/cinder/conversion/image_fetch_b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e_s84jq2k_ncdv-master-1@ceph, size 700.00 MB, duration 3.21 sec, destination /opt/stack/data/cinder/conversion/tmpyt28ab82 {{(pid=123870) _convert_image /opt/stack/cinder/cinder/image/image_utils.py:414}} May 14 21:17:26.269354 ncdv-master-1 cinder-volume[123870]: INFO cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Converted 700.00 MB image at 217.77 MB/s May 14 21:17:26.269354 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:17:26.503015 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:17:26.509409 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:17:26.656534 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] deleting rbd volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:17:28.823376 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-7b363af3-3b5b-4d67-bbfa-6d857475ae5d req-9541a140-2a04-4381-b082-176cfb51ad59 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:29.791163 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-7b363af3-3b5b-4d67-bbfa-6d857475ae5d req-9541a140-2a04-4381-b082-176cfb51ad59 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.968s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:29.800399 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7b363af3-3b5b-4d67-bbfa-6d857475ae5d req-9541a140-2a04-4381-b082-176cfb51ad59 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '2a72aa90-a850-47be-bc0c-572dbc34980a', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:17:32.267778 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-7b363af3-3b5b-4d67-bbfa-6d857475ae5d req-9541a140-2a04-4381-b082-176cfb51ad59 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Connection info returned from driver {'name': 'volumes/volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2a72aa90-a850-47be-bc0c-572dbc34980a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ae78095f-997a-436a-a1e6-99a2ad2be827'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:17:32.610287 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-7b363af3-3b5b-4d67-bbfa-6d857475ae5d req-9541a140-2a04-4381-b082-176cfb51ad59 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] attachment_update completed successfully. May 14 21:17:33.308291 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpyt28ab82 volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:44.648384 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-db49147d-008a-467e-b66f-f6c9be819186 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:17:45.427221 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-db49147d-008a-467e-b66f-f6c9be819186 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.779s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:17:45.429287 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-db49147d-008a-467e-b66f-f6c9be819186 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '2a72aa90-a850-47be-bc0c-572dbc34980a', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:17:45.508195 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-db49147d-008a-467e-b66f-f6c9be819186 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Connection info returned from driver {'name': 'volumes/volume-2a72aa90-a850-47be-bc0c-572dbc34980a', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2a72aa90-a850-47be-bc0c-572dbc34980a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '68dec14c-acb0-4186-8b13-9bbccfde9123'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:17:45.628084 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-db49147d-008a-467e-b66f-f6c9be819186 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] attachment_update completed successfully. May 14 21:18:02.472201 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:18:02.474944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:18:02.546873 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:18:02.580649 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:18:04.029977 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-4142f8f0-3d6d-4a62-9409-d4f3f4083f55 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Terminate volume connection completed successfully. May 14 21:18:04.031689 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-00bd8309-3eb2-4f98-aa7d-36085194315f req-4142f8f0-3d6d-4a62-9409-d4f3f4083f55 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Deleting attachment ae78095f-997a-436a-a1e6-99a2ad2be827. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:18:14.580607 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d9a2cb51-64c7-4073-a9a0-f5a177248b6f req-6238c344-b26a-48e6-92cd-189f43d1bb61 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Terminate volume connection completed successfully. May 14 21:18:14.580607 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d9a2cb51-64c7-4073-a9a0-f5a177248b6f req-6238c344-b26a-48e6-92cd-189f43d1bb61 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Deleting attachment 68dec14c-acb0-4186-8b13-9bbccfde9123. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:18:17.995857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:18:18.057515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Flow 'volume_create_manager' (0e1f0325-6758-49de-a16a-bd3064101177) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:18:18.068004 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (183fc77b-4a7f-4a85-824b-7edc99ea5eb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:18.100437 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (183fc77b-4a7f-4a85-824b-7edc99ea5eb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:18:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1811607681',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bed49647-1dd3-42d1-9736-4e72f9fde77c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='74bb7b020fae4bcaa3a440acc60c7a1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:18:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:18:18Z,use_quota=True,user_id='cfcb989a491440b3bf74ddc055ee14d3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:18.163160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bde54282-f9b2-4bc4-b615-8ae5a2fcb7cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:18.201113 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bde54282-f9b2-4bc4-b615-8ae5a2fcb7cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:18.279455 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (81597e39-0428-45f2-b822-211a10c38f57) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:18.501629 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (81597e39-0428-45f2-b822-211a10c38f57) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'bed49647-1dd3-42d1-9736-4e72f9fde77c', 'volume_name': 'volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:18.508442 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9530aff9-8c62-4344-bd23-7c0bde87b17b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:18.513444 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9530aff9-8c62-4344-bd23-7c0bde87b17b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:18.518840 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9ad4800c-a986-4082-8dfd-21a8b020e6fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:18.520350 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume bed49647-1dd3-42d1-9736-4e72f9fde77c: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:18:18.521309 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Cloning bed49647-1dd3-42d1-9736-4e72f9fde77c from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:18:28.913355 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] CMD "rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpyt28ab82 volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 55.606s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:18:29.086952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:18:30.318046 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:18:30.318726 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:18:30.318854 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Downloaded image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e ((None, None)) to volume a73964bc-4e69-4eef-b351-fcd6ec89ade1 successfully. {{(pid=123870) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1219}} May 14 21:18:30.409630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Temporary image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e for user 038739f45c3d4ab09457d6eed20b41d4 is deleted. {{(pid=123870) fetch /opt/stack/cinder/cinder/image/image_utils.py:1094}} May 14 21:18:30.578229 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota. May 14 21:18:30.580048 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota. May 14 21:18:30.967670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Created reservations ['208ab11c-a3a6-485b-8d0e-7c5415bfc0fa', 'b513788a-9c01-49d4-97c2-99674baaa2f6', 'ced93198-a77b-4670-97c2-f52ab8d75603', '059c801e-c95d-414f-b021-7ff5e49ab3f1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:18:30.989319 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Creating image volume entry: {'group_id': None, 'cluster_name': None, 'service_uuid': None, 'provider_location': None, 'size': 1, 'ec2_id': None, 'project_id': '0911a024287e430ea6ee015389f0c440', 'host': 'ncdv-master-1@ceph#ceph', 'user_id': '038739f45c3d4ab09457d6eed20b41d4', 'shared_targets': True, 'metadata': {}, 'encryption_key_id': None, 'previous_status': None, 'provider_geometry': None, 'deleted': False, 'volume_metadata': [], 'multiattach': False, 'provider_auth': None, 'created_at': datetime.datetime(2022, 5, 14, 21, 16, 46, tzinfo=datetime.timezone.utc), 'display_description': None, 'volume_type_id': '0a4caf03-4613-4ca9-9e3b-1297a183a0fc', 'replication_status': None, 'replication_extended_status': None, 'bootable': False, 'updated_at': datetime.datetime(2022, 5, 14, 21, 16, 47, tzinfo=datetime.timezone.utc), 'provider_id': None, 'display_name': 'image-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'deleted_at': None, 'volume_admin_metadata': [], 'volume_glance_metadata': [{'key': 'signature_verified', 'value': 'False'}], 'launched_at': None, 'scheduled_at': datetime.datetime(2022, 5, 14, 21, 16, 46, tzinfo=datetime.timezone.utc), 'availability_zone': 'nova', 'replication_driver_data': None, 'consistencygroup_id': None, 'terminated_at': None, 'attach_status': 'detached', 'status': 'creating', 'source_volid': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1'}. {{(pid=123870) _clone_image_volume /opt/stack/cinder/cinder/volume/manager.py:1616}} May 14 21:18:31.386548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume reschedule parameters: False retry: None {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:18:31.414686 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Flow 'volume_create_manager' (8b05b6f5-b8c3-4cc9-9427-fe60a6c0eefa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:18:31.421931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (26dc12db-e3f9-4e4e-8cf0-3c31a91fe35d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:31.465234 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (26dc12db-e3f9-4e4e-8cf0-3c31a91fe35d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:16:46Z,deleted=False,deleted_at=None,display_description=None,display_name='image-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=59ff51c3-dc23-4af9-af80-0fb5c11ea498,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0911a024287e430ea6ee015389f0c440',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:16:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=a73964bc-4e69-4eef-b351-fcd6ec89ade1,status='creating',terminated_at=None,updated_at=2022-05-14T21:16:47Z,use_quota=True,user_id='038739f45c3d4ab09457d6eed20b41d4',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:31.471817 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (977bcdf8-f9e6-434c-8ecc-4b2fab63cceb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:31.475967 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (977bcdf8-f9e6-434c-8ecc-4b2fab63cceb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:31.481520 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae44c28f-3ed1-48be-b805-4fe4fd898bc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:31.516995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae44c28f-3ed1-48be-b805-4fe4fd898bc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'volume_name': 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'volume_size': 1, 'source_volid': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'source_volstatus': 'downloading'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:31.523377 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0e7b3469-6428-4388-b7ad-95d9ba1e3fc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:31.541966 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0e7b3469-6428-4388-b7ad-95d9ba1e3fc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:31.550282 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (14d3bf4a-606f-4ca9-8758-740672780af6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:31.556713 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'volume_size': 1, 'source_volid': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'source_volstatus': 'downloading'} May 14 21:18:31.577621 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:18:31.806443 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] creating snapshot='volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:18:33.440771 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] cloning 'volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1@volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498.clone_snap' to 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:18:35.230010 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:18:35.231184 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:18:35.231650 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:18:35.701585 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:18:35.705954 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (14d3bf4a-606f-4ca9-8758-740672780af6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'volume_size': 1, 'source_volid': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'source_volstatus': 'downloading'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:35.737884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (40a8baa6-0ddb-4d5f-8936-e0b4ada1fc1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:35.815391 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498 (59ff51c3-dc23-4af9-af80-0fb5c11ea498): created successfully May 14 21:18:35.820804 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (40a8baa6-0ddb-4d5f-8936-e0b4ada1fc1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:35.828608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Flow 'volume_create_manager' (8b05b6f5-b8c3-4cc9-9427-fe60a6c0eefa) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:18:35.924347 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Created volume successfully. May 14 21:18:36.160811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Creating new image-volume cache entry for image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on ncdv-master-1@ceph#ceph {{(pid=123870) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:104}} May 14 21:18:36.253112 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] New image-volume cache entry created: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 18, 36, 163925)}. {{(pid=123870) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:129}} May 14 21:18:36.258238 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 109.115s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:18:36.258831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Marking volume a73964bc-4e69-4eef-b351-fcd6ec89ade1 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:18:36.348178 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to a73964bc-4e69-4eef-b351-fcd6ec89ade1. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:18:36.349099 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Creating volume glance metadata for volume a73964bc-4e69-4eef-b351-fcd6ec89ade1 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:18:36.505537 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b598e34-1dba-48c5-b859-7c4673eee718) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:36.514284 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c6d40391-acad-4b06-829c-8d6c7b8e92d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:36.622410 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 (a73964bc-4e69-4eef-b351-fcd6ec89ade1): created successfully May 14 21:18:36.627597 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c6d40391-acad-4b06-829c-8d6c7b8e92d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:36.637489 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Flow 'volume_create_manager' (3ab48f9e-cbd9-4d68-b60c-e75c6578a9ca) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:18:36.781423 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 76.244s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:18:36.795717 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-76d4f63e-bece-4d4d-87ba-d4695a7ae9c8 tempest-LiveMigrationTest-772526 None] Created volume successfully. May 14 21:18:36.977799 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 18, 36, 878166)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:18:36.978556 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:18:36.979155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:18:36.984355 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.203s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:18:36.985849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:18:36.998136 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 18, 36, 989652)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:18:36.998844 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:18:36.999253 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:18:37.025246 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:18:37.272827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] creating snapshot='volume-022c11da-58a1-4855-977c-de51c3870d3f.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:18:39.092800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-022c11da-58a1-4855-977c-de51c3870d3f.clone_snap' to 'volume-022c11da-58a1-4855-977c-de51c3870d3f' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:18:39.154913 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 20.632s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:18:39.246281 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 18, 39, 162807)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:18:39.246496 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:18:39.246962 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:18:39.259334 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.104s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:18:39.260263 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:18:39.274599 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 18, 39, 264893)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:18:39.274599 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:18:39.274599 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:18:39.295335 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:18:39.660763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] creating snapshot='volume-bed49647-1dd3-42d1-9736-4e72f9fde77c.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:18:42.036601 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-bed49647-1dd3-42d1-9736-4e72f9fde77c.clone_snap' to 'volume-bed49647-1dd3-42d1-9736-4e72f9fde77c' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:18:42.111760 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:18:42.112798 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:18:42.113181 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:18:43.030768 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:18:43.035028 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Marking volume 022c11da-58a1-4855-977c-de51c3870d3f as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:18:43.112246 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 022c11da-58a1-4855-977c-de51c3870d3f. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:18:43.112761 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Creating volume glance metadata for volume 022c11da-58a1-4855-977c-de51c3870d3f backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:18:43.340767 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6c381059-779e-41b3-932c-06d8fe64a40b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-022c11da-58a1-4855-977c-de51c3870d3f', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:43.356969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ee6a65eb-170d-453e-ac81-520995068d81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:43.491079 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Volume volume-022c11da-58a1-4855-977c-de51c3870d3f (022c11da-58a1-4855-977c-de51c3870d3f): created successfully May 14 21:18:43.495158 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ee6a65eb-170d-453e-ac81-520995068d81) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:43.501442 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Flow 'volume_create_manager' (26f9fbc0-f915-4194-96bb-9b4f80bef200) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:18:43.591182 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0cbe01f2-ccb3-49b9-a6ab-f7363857fd8d tempest-ServersTestBootFromVolume-823166955 None] Created volume successfully. May 14 21:18:43.860114 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:18:43.861929 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:18:43.863557 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:18:44.037657 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:18:44.038267 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Marking volume bed49647-1dd3-42d1-9736-4e72f9fde77c as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:18:44.153240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to bed49647-1dd3-42d1-9736-4e72f9fde77c. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:18:44.156322 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Creating volume glance metadata for volume bed49647-1dd3-42d1-9736-4e72f9fde77c backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:18:44.378273 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9ad4800c-a986-4082-8dfd-21a8b020e6fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:44.384820 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (22976de0-cac7-4418-bde3-861c5df5bcf8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:18:44.493577 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume volume-bed49647-1dd3-42d1-9736-4e72f9fde77c (bed49647-1dd3-42d1-9736-4e72f9fde77c): created successfully May 14 21:18:44.499279 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (22976de0-cac7-4418-bde3-861c5df5bcf8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:18:44.508441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Flow 'volume_create_manager' (0e1f0325-6758-49de-a16a-bd3064101177) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:18:44.633114 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-25ce598c-1503-4032-a3aa-f270661b06b9 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Created volume successfully. May 14 21:18:45.725882 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-25b3dd5c-e30a-4764-85eb-55b47803a5d5 req-31a79dd1-4168-4bf1-a3ab-ffbb97063454 tempest-LiveMigrationTest-772526 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:18:46.868348 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-25b3dd5c-e30a-4764-85eb-55b47803a5d5 req-31a79dd1-4168-4bf1-a3ab-ffbb97063454 tempest-LiveMigrationTest-772526 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.142s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:18:46.869174 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-25b3dd5c-e30a-4764-85eb-55b47803a5d5 req-31a79dd1-4168-4bf1-a3ab-ffbb97063454 tempest-LiveMigrationTest-772526 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:18:47.044076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-25b3dd5c-e30a-4764-85eb-55b47803a5d5 req-31a79dd1-4168-4bf1-a3ab-ffbb97063454 tempest-LiveMigrationTest-772526 None] Connection info returned from driver {'name': 'volumes/volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0a12c53a-ae21-4216-8cb2-bbc4e03b11d5'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:18:47.234488 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-25b3dd5c-e30a-4764-85eb-55b47803a5d5 req-31a79dd1-4168-4bf1-a3ab-ffbb97063454 tempest-LiveMigrationTest-772526 None] attachment_update completed successfully. May 14 21:18:52.294092 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-9269c383-7dd2-47c4-a0b4-89defdb587e7 req-ff7bfdd1-c56c-46be-9e3a-e656c6b09f34 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:18:52.698560 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-7f0d01a8-047b-4353-8f17-d8c9a4c59980 req-9832d40d-5a1c-4297-bb1c-b1aa66d08f5b tempest-ServersTestBootFromVolume-823166955 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:18:53.831008 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-9269c383-7dd2-47c4-a0b4-89defdb587e7 req-ff7bfdd1-c56c-46be-9e3a-e656c6b09f34 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.537s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:18:53.833307 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-9269c383-7dd2-47c4-a0b4-89defdb587e7 req-ff7bfdd1-c56c-46be-9e3a-e656c6b09f34 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'bed49647-1dd3-42d1-9736-4e72f9fde77c', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:18:53.879253 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-7f0d01a8-047b-4353-8f17-d8c9a4c59980 req-9832d40d-5a1c-4297-bb1c-b1aa66d08f5b tempest-ServersTestBootFromVolume-823166955 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.181s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:18:53.880031 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7f0d01a8-047b-4353-8f17-d8c9a4c59980 req-9832d40d-5a1c-4297-bb1c-b1aa66d08f5b tempest-ServersTestBootFromVolume-823166955 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-022c11da-58a1-4855-977c-de51c3870d3f', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '022c11da-58a1-4855-977c-de51c3870d3f', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:18:53.918095 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-9269c383-7dd2-47c4-a0b4-89defdb587e7 req-ff7bfdd1-c56c-46be-9e3a-e656c6b09f34 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Connection info returned from driver {'name': 'volumes/volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bed49647-1dd3-42d1-9736-4e72f9fde77c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7aefa2e3-98d2-4660-9d49-a89e1205a6c0'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:18:54.015132 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-7f0d01a8-047b-4353-8f17-d8c9a4c59980 req-9832d40d-5a1c-4297-bb1c-b1aa66d08f5b tempest-ServersTestBootFromVolume-823166955 None] Connection info returned from driver {'name': 'volumes/volume-022c11da-58a1-4855-977c-de51c3870d3f', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '022c11da-58a1-4855-977c-de51c3870d3f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ed42ed9b-ba24-462c-bad2-ac06362b22ee'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:18:54.085355 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-9269c383-7dd2-47c4-a0b4-89defdb587e7 req-ff7bfdd1-c56c-46be-9e3a-e656c6b09f34 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] attachment_update completed successfully. May 14 21:18:54.187821 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-7f0d01a8-047b-4353-8f17-d8c9a4c59980 req-9832d40d-5a1c-4297-bb1c-b1aa66d08f5b tempest-ServersTestBootFromVolume-823166955 None] attachment_update completed successfully. May 14 21:19:02.469232 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:19:02.471283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:02.560996 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:02.628761 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:19:19.810989 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-2b631f19-d6ba-46ab-8da9-51b2a9db0958 tempest-LiveMigrationTest-151813528 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:19:21.306030 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-2b631f19-d6ba-46ab-8da9-51b2a9db0958 tempest-LiveMigrationTest-151813528 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.495s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:19:21.310875 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-2b631f19-d6ba-46ab-8da9-51b2a9db0958 tempest-LiveMigrationTest-151813528 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:19:21.474798 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-2b631f19-d6ba-46ab-8da9-51b2a9db0958 tempest-LiveMigrationTest-151813528 None] Connection info returned from driver {'name': 'volumes/volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a73964bc-4e69-4eef-b351-fcd6ec89ade1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'dc143042-7f37-44a3-9487-3cd4e24b0e5c'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:19:21.725363 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-2b631f19-d6ba-46ab-8da9-51b2a9db0958 tempest-LiveMigrationTest-151813528 None] attachment_update completed successfully. May 14 21:19:24.382217 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-e8dc233a-dc93-4e0e-b284-c13132752b47 tempest-LiveMigrationTest-151813528 None] Terminate volume connection completed successfully. May 14 21:19:24.385057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f45320ec-dba4-49dd-a4d4-735f528a50dc req-e8dc233a-dc93-4e0e-b284-c13132752b47 tempest-LiveMigrationTest-151813528 None] Deleting attachment dc143042-7f37-44a3-9487-3cd4e24b0e5c. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:19:26.230841 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-44e63b03-209b-4de3-b42d-1ad7fcbbefd0 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:19:27.349956 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-44e63b03-209b-4de3-b42d-1ad7fcbbefd0 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.119s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:19:27.350885 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-44e63b03-209b-4de3-b42d-1ad7fcbbefd0 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'bed49647-1dd3-42d1-9736-4e72f9fde77c', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:19:27.548611 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-44e63b03-209b-4de3-b42d-1ad7fcbbefd0 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Connection info returned from driver {'name': 'volumes/volume-bed49647-1dd3-42d1-9736-4e72f9fde77c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bed49647-1dd3-42d1-9736-4e72f9fde77c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'cb5ac0c0-1619-46a4-b6e1-09602d2957b4'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:19:27.863908 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-44e63b03-209b-4de3-b42d-1ad7fcbbefd0 tempest-LiveAutoBlockMigrationV225Test-1177449053 None] attachment_update completed successfully. May 14 21:19:34.031095 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-fafed17f-0315-4716-9629-778aecfb5cfc req-bd0044be-f704-4d66-ba45-64bb0d9a073d tempest-LiveMigrationTest-772526 None] Terminate volume connection completed successfully. May 14 21:19:34.031724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-fafed17f-0315-4716-9629-778aecfb5cfc req-bd0044be-f704-4d66-ba45-64bb0d9a073d tempest-LiveMigrationTest-772526 None] Deleting attachment 0a12c53a-ae21-4216-8cb2-bbc4e03b11d5. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:19:34.607574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] Lock "b'cinder-a73964bc-4e69-4eef-b351-fcd6ec89ade1-delete_volume'" acquired by "delete_volume" :: waited 0.027s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:19:34.755272 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:34.926016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:19:34.926626 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] volume has clone snapshot(s) {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1290}} May 14 21:19:34.928030 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:19:36.677687 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] Created reservations ['038542ab-ca5b-413e-8682-7e8edd59a1bb', '3b29e0b1-636a-4c68-a34d-ccd108b19f30', 'd70aafd2-62bc-4464-91b0-80b843c9348c', 'c990df00-4e5c-4d2a-a90f-826c93cd27aa'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:19:36.798780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] Lock "b'cinder-e0863bd3-bff3-4125-b857-a7cf7b43cf86-delete_volume'" acquired by "delete_volume" :: waited 0.039s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:19:36.909524 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:37.073235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:19:37.074513 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] Volume volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:19:37.200002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] deleting rbd volume volume-e0863bd3-bff3-4125-b857-a7cf7b43cf86 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:19:37.266177 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:37.291634 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:37.317203 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:19:37.322674 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] Deleted volume successfully. May 14 21:19:37.512802 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-fafed17f-0315-4716-9629-778aecfb5cfc req-5efb38e3-9367-41d3-bcda-285f28ccce3f tempest-LiveMigrationTest-772526 None] Lock "b'cinder-a73964bc-4e69-4eef-b351-fcd6ec89ade1-delete_volume'" released by "delete_volume" :: held 2.906s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:19:42.356638 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-dbcce001-797b-44f0-9a73-38d26946451d tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Terminate volume connection completed successfully. May 14 21:19:42.357361 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-fd131a6e-62ed-4bd0-bb59-f12b70cd0c81 req-dbcce001-797b-44f0-9a73-38d26946451d tempest-LiveAutoBlockMigrationV225Test-1177449053 None] Deleting attachment 7aefa2e3-98d2-4660-9d49-a89e1205a6c0. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:19:44.971004 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] Created reservations ['5151e9a9-1e39-496c-962d-aac8f6ef3d02', '4a2cf9dd-ad4f-4e1e-b89c-db6418c69236', 'f2223fc9-325e-4f1e-a4ec-afe7da6d1e1c', 'd42dcdad-b8b1-4776-8cae-625f4264688a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:19:45.232884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:45.306496 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:45.477020 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:19:45.487419 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] Deleted volume successfully. May 14 21:19:45.581127 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ee05422c-79d7-44e0-9976-c04527bf3dec tempest-LiveMigrationTest-772526 None] Lock "b'cinder-e0863bd3-bff3-4125-b857-a7cf7b43cf86-delete_volume'" released by "delete_volume" :: held 8.780s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:19:52.810525 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-ae9aceab-baf3-4bbf-83eb-0a119a550acc tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Terminate volume connection completed successfully. May 14 21:19:52.821859 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-ae9aceab-baf3-4bbf-83eb-0a119a550acc tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Deleting attachment cb5ac0c0-1619-46a4-b6e1-09602d2957b4. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:19:53.453057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Lock "b'cinder-bed49647-1dd3-42d1-9736-4e72f9fde77c-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:19:53.636062 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:53.949844 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:19:54.249469 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] deleting rbd volume volume-bed49647-1dd3-42d1-9736-4e72f9fde77c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:19:55.719724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Lock "b'cinder-2a72aa90-a850-47be-bc0c-572dbc34980a-delete_volume'" acquired by "delete_volume" :: waited 0.028s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:19:55.876821 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:19:56.077388 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:19:56.078137 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Volume volume-2a72aa90-a850-47be-bc0c-572dbc34980a is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:19:56.319874 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] deleting rbd volume volume-2a72aa90-a850-47be-bc0c-572dbc34980a {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:19:58.840970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:19:58.863033 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Flow 'volume_create_manager' (e5424e95-a56c-4d11-93ba-62959c3160ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:19:58.869555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c4dd3c41-18bc-4d1d-b0cc-2b5fa48d1175) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:19:58.885491 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c4dd3c41-18bc-4d1d-b0cc-2b5fa48d1175) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:19:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-DeleteServersTestJSON-volume-26704029',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=962a3525-29b8-4d27-b336-6be335779521,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b031454047774b4fb8b459f226f645d2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:19:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:19:59Z,use_quota=True,user_id='71080496c33c4e2c8b2e16b950f0b9ea',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:19:58.891120 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (750c29b2-1c66-4326-bca7-10ffa3c31c9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:19:58.899424 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (750c29b2-1c66-4326-bca7-10ffa3c31c9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:19:58.901840 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ebab8bce-8752-4ffa-ba43-5f242b353455) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:19:58.906497 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ebab8bce-8752-4ffa-ba43-5f242b353455) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '962a3525-29b8-4d27-b336-6be335779521', 'volume_name': 'volume-962a3525-29b8-4d27-b336-6be335779521', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:19:58.912120 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (abad95e7-0a84-4429-bfb9-73e1609e5c32) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:19:58.916009 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (abad95e7-0a84-4429-bfb9-73e1609e5c32) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:19:58.922028 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c77937e6-d93c-49cf-a978-e37e516f9ba6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:19:58.925554 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Volume 962a3525-29b8-4d27-b336-6be335779521: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-962a3525-29b8-4d27-b336-6be335779521', 'volume_size': 1} May 14 21:19:58.930679 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] creating volume 'volume-962a3525-29b8-4d27-b336-6be335779521' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:19:58.930679 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:00.173726 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:20:00.175277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:20:00.182810 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c77937e6-d93c-49cf-a978-e37e516f9ba6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-962a3525-29b8-4d27-b336-6be335779521', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:20:00.188081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bc772672-d2c0-445f-b2e7-99a259b671b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:20:00.300729 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Volume volume-962a3525-29b8-4d27-b336-6be335779521 (962a3525-29b8-4d27-b336-6be335779521): created successfully May 14 21:20:00.305622 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bc772672-d2c0-445f-b2e7-99a259b671b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:20:00.316678 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Flow 'volume_create_manager' (e5424e95-a56c-4d11-93ba-62959c3160ef) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:20:00.417730 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-789e2943-ce63-46a3-ac92-923fa089665d tempest-DeleteServersTestJSON-1962247194 None] Created volume successfully. May 14 21:20:02.132588 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-208da729-6e02-493a-b978-f998a821cda1 req-6307124f-eda3-468b-ba0d-317154d0df6c tempest-DeleteServersTestJSON-1962247194 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:20:02.469034 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:20:02.470232 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:02.517609 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:02.595271 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:20:03.099976 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-208da729-6e02-493a-b978-f998a821cda1 req-6307124f-eda3-468b-ba0d-317154d0df6c tempest-DeleteServersTestJSON-1962247194 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.967s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:20:03.100632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-208da729-6e02-493a-b978-f998a821cda1 req-6307124f-eda3-468b-ba0d-317154d0df6c tempest-DeleteServersTestJSON-1962247194 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-962a3525-29b8-4d27-b336-6be335779521', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '962a3525-29b8-4d27-b336-6be335779521', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:20:03.239134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-208da729-6e02-493a-b978-f998a821cda1 req-6307124f-eda3-468b-ba0d-317154d0df6c tempest-DeleteServersTestJSON-1962247194 None] Connection info returned from driver {'name': 'volumes/volume-962a3525-29b8-4d27-b336-6be335779521', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '962a3525-29b8-4d27-b336-6be335779521', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '836a3f5d-1184-43f1-823c-27590d154e88'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:20:03.396010 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-208da729-6e02-493a-b978-f998a821cda1 req-6307124f-eda3-468b-ba0d-317154d0df6c tempest-DeleteServersTestJSON-1962247194 None] attachment_update completed successfully. May 14 21:20:03.879426 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:20:04.156901 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] deleting parent snapshot volume-bed49647-1dd3-42d1-9736-4e72f9fde77c.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:20:08.744047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Created reservations ['f26ab044-bb26-4156-ba35-1a3fa9370dc3', '2240d3d3-9f40-44a6-81a7-219e7425af86', '4ce5481e-b8a7-457b-b3df-18c2b86fdcd4', '6c55c2df-b7b7-44c9-aefd-9be87096e7e5'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:20:08.929890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Created reservations ['0575b52d-55c7-47ee-b03a-b349f294b910', '8f841bec-c5aa-44af-8fef-71794110c152', 'd61133a4-ed1a-4967-bf27-8d37487be64d', '18bda774-ac6b-4978-96ab-0ae7906593f6'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:20:09.117583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:09.235727 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:09.368152 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:20:09.373531 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Deleted volume successfully. May 14 21:20:09.517175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-bf387ae4-3b96-4c30-bcae-cac9077aec84 req-27e75b68-a5f2-49bc-89c9-278a3d2d1db0 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Lock "b'cinder-bed49647-1dd3-42d1-9736-4e72f9fde77c-delete_volume'" released by "delete_volume" :: held 16.061s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:20:09.656509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:09.720029 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:20:09.810892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:20:09.820364 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Deleted volume successfully. May 14 21:20:09.904081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9bb9d47f-ec17-4877-bd15-dfac3f64a433 tempest-LiveAutoBlockMigrationV225Test-1811005538 None] Lock "b'cinder-2a72aa90-a850-47be-bc0c-572dbc34980a-delete_volume'" released by "delete_volume" :: held 14.180s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:20:09.963017 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-b0420979-15f3-414c-9985-bbad51c20edd req-bda28030-f696-49ae-8874-dabfeddac961 tempest-DeleteServersTestJSON-1962247194 None] Terminate volume connection completed successfully. May 14 21:20:09.963441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-b0420979-15f3-414c-9985-bbad51c20edd req-bda28030-f696-49ae-8874-dabfeddac961 tempest-DeleteServersTestJSON-1962247194 None] Deleting attachment 836a3f5d-1184-43f1-823c-27590d154e88. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:21:02.472948 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:21:02.473778 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:21:02.602154 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:21:02.717057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:21:53.130268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:21:53.169515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Flow 'volume_create_manager' (bfa8bb2d-1000-4eab-bc71-32b6eed649d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:21:53.198405 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c00aaafe-e6e7-46e9-bd8f-9eee62beada2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:53.252239 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c00aaafe-e6e7-46e9-bd8f-9eee62beada2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:21:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1416303206',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=40376eea-0c0f-488c-ae00-97b792b66c72,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d3f937d0cc40e6b1cd90e72b6fb07e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:21:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:21:53Z,use_quota=True,user_id='2d0ac47523ba4f288bcd89f8228ef910',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:53.261160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2d6d4c4f-ad3d-42bc-8cad-a818be1c74d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:53.269140 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2d6d4c4f-ad3d-42bc-8cad-a818be1c74d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:53.285879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ea4ef796-25cb-4518-9eac-90c12000bfde) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:53.453009 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ea4ef796-25cb-4518-9eac-90c12000bfde) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '40376eea-0c0f-488c-ae00-97b792b66c72', 'volume_name': 'volume-40376eea-0c0f-488c-ae00-97b792b66c72', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:53.471154 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3f456afc-43d2-4a3e-bb3a-6d7634fba21e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:53.482687 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3f456afc-43d2-4a3e-bb3a-6d7634fba21e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:53.497436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b88efea-7fb2-45f9-aa41-bc1211620f31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:53.505381 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume 40376eea-0c0f-488c-ae00-97b792b66c72: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-40376eea-0c0f-488c-ae00-97b792b66c72', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:21:53.506297 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Cloning 40376eea-0c0f-488c-ae00-97b792b66c72 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:21:53.517506 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:21:53.587059 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 21, 53, 523324)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:21:53.588178 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:21:53.588947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:21:53.596534 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.078s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:21:53.598461 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:21:53.609368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 21, 53, 605431)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:21:53.610623 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:21:53.611334 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:21:53.628839 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:21:53.986474 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] creating snapshot='volume-40376eea-0c0f-488c-ae00-97b792b66c72.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:21:56.556718 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-40376eea-0c0f-488c-ae00-97b792b66c72.clone_snap' to 'volume-40376eea-0c0f-488c-ae00-97b792b66c72' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:21:56.689373 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:21:56.737976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Flow 'volume_create_manager' (e497ab0f-ee6b-43f4-9a53-2e593eea2bd8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:21:56.760073 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (44cdcb2d-b07c-4572-b8e4-953e58c51b80) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:56.794531 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (44cdcb2d-b07c-4572-b8e4-953e58c51b80) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:21:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-29733319',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2aecd011-8bb0-4a67-b8ce-def64dd1d427,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cabbde4fba484c6d83f6ae32716cf82b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:21:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:21:56Z,use_quota=True,user_id='07186d1e535e4de2ae6b8269ca10b4c6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:56.800318 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ac6924c9-146f-422d-a5ed-db0d50734c34) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:56.805381 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ac6924c9-146f-422d-a5ed-db0d50734c34) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:56.819288 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (01d60e12-7b78-439d-b46c-ef7c71ecae24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:56.838623 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (01d60e12-7b78-439d-b46c-ef7c71ecae24) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'volume_name': 'volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:56.855687 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0f91ccd2-01c3-4df5-a274-2b507c5f98bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:56.862213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0f91ccd2-01c3-4df5-a274-2b507c5f98bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:56.894101 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4522e214-738f-4bd8-ad67-57802db45417) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:56.898455 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Volume 2aecd011-8bb0-4a67-b8ce-def64dd1d427: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'volume_size': 1} May 14 21:21:56.898959 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] creating volume 'volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:21:56.899887 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:21:58.528180 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:21:58.528821 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:21:58.532575 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4522e214-738f-4bd8-ad67-57802db45417) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:58.538038 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1bf3f5c2-f8a9-4142-8cb7-e7712e34586a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:21:58.602218 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Volume volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427 (2aecd011-8bb0-4a67-b8ce-def64dd1d427): created successfully May 14 21:21:58.606280 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1bf3f5c2-f8a9-4142-8cb7-e7712e34586a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:21:58.614192 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Flow 'volume_create_manager' (e497ab0f-ee6b-43f4-9a53-2e593eea2bd8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:21:58.693728 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-decfedf6-2a4c-4442-a56f-bed8c0ccc6f2 tempest-VolumesAdminNegativeTest-2091123030 None] Created volume successfully. May 14 21:21:59.272283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:21:59.273431 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:21:59.273899 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:21:59.968041 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:21:59.968706 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Marking volume 40376eea-0c0f-488c-ae00-97b792b66c72 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:22:00.053894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 40376eea-0c0f-488c-ae00-97b792b66c72. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:22:00.054312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Creating volume glance metadata for volume 40376eea-0c0f-488c-ae00-97b792b66c72 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:22:00.131076 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-cbab3b6d-d9f0-4460-8d58-91b18d2c76ef req-8c633406-7d27-40ca-8c36-f5bae3a36363 tempest-VolumesAdminNegativeTest-2091123030 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:22:00.247952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b88efea-7fb2-45f9-aa41-bc1211620f31) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-40376eea-0c0f-488c-ae00-97b792b66c72', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:00.253944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1b9a27bf-595b-4e10-999c-e1881cda6fe2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:00.328021 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume volume-40376eea-0c0f-488c-ae00-97b792b66c72 (40376eea-0c0f-488c-ae00-97b792b66c72): created successfully May 14 21:22:00.332091 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1b9a27bf-595b-4e10-999c-e1881cda6fe2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:00.340845 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Flow 'volume_create_manager' (bfa8bb2d-1000-4eab-bc71-32b6eed649d6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:00.403458 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-51eaa449-eea4-4fb0-8c71-62c6a9a49004 tempest-TaggedBootDevicesTest_v242-387890351 None] Created volume successfully. May 14 21:22:01.496488 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-cbab3b6d-d9f0-4460-8d58-91b18d2c76ef req-8c633406-7d27-40ca-8c36-f5bae3a36363 tempest-VolumesAdminNegativeTest-2091123030 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.365s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:22:01.497149 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cbab3b6d-d9f0-4460-8d58-91b18d2c76ef req-8c633406-7d27-40ca-8c36-f5bae3a36363 tempest-VolumesAdminNegativeTest-2091123030 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:22:01.807472 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-cbab3b6d-d9f0-4460-8d58-91b18d2c76ef req-8c633406-7d27-40ca-8c36-f5bae3a36363 tempest-VolumesAdminNegativeTest-2091123030 None] Connection info returned from driver {'name': 'volumes/volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2aecd011-8bb0-4a67-b8ce-def64dd1d427', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'deb159b5-0a76-4ba9-b05f-ba47c239c49d'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:22:01.991578 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-cbab3b6d-d9f0-4460-8d58-91b18d2c76ef req-8c633406-7d27-40ca-8c36-f5bae3a36363 tempest-VolumesAdminNegativeTest-2091123030 None] attachment_update completed successfully. May 14 21:22:02.004418 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:22:02.026993 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Flow 'volume_create_manager' (a92e02c0-ba7c-48c1-931f-aecec31ed195) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:02.034091 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1eb7d06-5b19-4a0a-9574-636baa7a3307) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:02.052150 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1eb7d06-5b19-4a0a-9574-636baa7a3307) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1238540227',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5ab5558f-588e-4b04-84f6-6f28c0048e77,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d3f937d0cc40e6b1cd90e72b6fb07e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:22:01Z,use_quota=True,user_id='2d0ac47523ba4f288bcd89f8228ef910',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:02.057666 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a50e402a-8b28-41cc-889c-1837380adde4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:02.062511 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a50e402a-8b28-41cc-889c-1837380adde4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:02.067830 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b8250a5c-4b57-468a-97b4-ddd7a462ae90) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:02.071995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b8250a5c-4b57-468a-97b4-ddd7a462ae90) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5ab5558f-588e-4b04-84f6-6f28c0048e77', 'volume_name': 'volume-5ab5558f-588e-4b04-84f6-6f28c0048e77', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:02.079396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (569879aa-9f3c-4199-8216-dad278b22425) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:02.085008 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (569879aa-9f3c-4199-8216-dad278b22425) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:02.088961 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (598d6d0b-4dae-48c3-aab3-03a45abe0b9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:02.090102 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume 5ab5558f-588e-4b04-84f6-6f28c0048e77: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5ab5558f-588e-4b04-84f6-6f28c0048e77', 'volume_size': 1} May 14 21:22:02.090731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] creating volume 'volume-5ab5558f-588e-4b04-84f6-6f28c0048e77' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:22:02.091349 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:02.475379 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:22:02.476242 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:02.498721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:02.522465 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:22:03.243519 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:22:03.243893 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:22:03.248338 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (598d6d0b-4dae-48c3-aab3-03a45abe0b9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5ab5558f-588e-4b04-84f6-6f28c0048e77', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:03.254224 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1b96f938-4b29-44fc-ab2c-90819f5331a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:03.322662 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume volume-5ab5558f-588e-4b04-84f6-6f28c0048e77 (5ab5558f-588e-4b04-84f6-6f28c0048e77): created successfully May 14 21:22:03.326552 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1b96f938-4b29-44fc-ab2c-90819f5331a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:03.333776 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Flow 'volume_create_manager' (a92e02c0-ba7c-48c1-931f-aecec31ed195) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:03.407327 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c41d2fd4-115e-4101-aa33-aef43746ab57 tempest-TaggedBootDevicesTest_v242-387890351 None] Created volume successfully. May 14 21:22:05.152244 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:22:05.172035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Flow 'volume_create_manager' (d968fa59-c1ec-48e6-b68c-246fbdd3579c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:05.178894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3726555d-c3d4-4414-908d-d8ac2a9d3182) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:05.194743 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3726555d-c3d4-4414-908d-d8ac2a9d3182) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:22:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1278668758',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d3f937d0cc40e6b1cd90e72b6fb07e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:22:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:22:05Z,use_quota=True,user_id='2d0ac47523ba4f288bcd89f8228ef910',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:05.200151 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6dd71250-8cc3-435d-846c-515158c26055) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:05.204313 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6dd71250-8cc3-435d-846c-515158c26055) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:05.209469 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ebfd4930-7ca1-434d-9153-f6af2a790530) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:05.213475 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ebfd4930-7ca1-434d-9153-f6af2a790530) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'volume_name': 'volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:05.218376 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (435bca5c-464c-4204-9c87-08c66eea4898) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:05.222277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (435bca5c-464c-4204-9c87-08c66eea4898) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:05.227010 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (15ae3c9e-1792-4f31-a5c5-4fe9352347f6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:05.228102 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'volume_size': 1} May 14 21:22:05.228489 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] creating volume 'volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:22:05.229159 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:06.365039 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:22:06.365644 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:22:06.369473 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (15ae3c9e-1792-4f31-a5c5-4fe9352347f6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:06.374788 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (558c1ffa-ffc7-47f2-af56-60982f70b8ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:06.499706 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db (e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db): created successfully May 14 21:22:06.503422 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (558c1ffa-ffc7-47f2-af56-60982f70b8ae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:06.509066 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Flow 'volume_create_manager' (d968fa59-c1ec-48e6-b68c-246fbdd3579c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:06.641239 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2507677e-95a2-4e85-ab22-817adfc3d1b9 tempest-TaggedBootDevicesTest_v242-387890351 None] Created volume successfully. May 14 21:22:06.999224 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-76b65b9b-fa48-419a-828b-3c81da5f6b5b req-f3639457-634f-4aaf-b35a-1244bbf171fe tempest-VolumesAdminNegativeTest-2091123030 None] Terminate volume connection completed successfully. May 14 21:22:07.002615 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-76b65b9b-fa48-419a-828b-3c81da5f6b5b req-f3639457-634f-4aaf-b35a-1244bbf171fe tempest-VolumesAdminNegativeTest-2091123030 None] Deleting attachment deb159b5-0a76-4ba9-b05f-ba47c239c49d. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:22:08.552155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:22:08.574679 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Flow 'volume_create_manager' (b17bc378-38e1-413b-9297-1e7d9bd015c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:08.587530 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0cccf282-aa7a-4436-b9b3-5736da8b6abe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:08.618337 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0cccf282-aa7a-4436-b9b3-5736da8b6abe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:22:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-1685589490',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=444d77c2-4003-4c0b-a856-d73268fac0ac,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cabbde4fba484c6d83f6ae32716cf82b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:22:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:22:08Z,use_quota=True,user_id='07186d1e535e4de2ae6b8269ca10b4c6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:08.632050 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (27711af3-f4d3-4e0f-b31e-a9cb0b94ace6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:08.640981 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (27711af3-f4d3-4e0f-b31e-a9cb0b94ace6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:08.647040 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5288bbfa-322f-4a2f-a4cb-e1519ffb4433) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:08.651721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5288bbfa-322f-4a2f-a4cb-e1519ffb4433) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '444d77c2-4003-4c0b-a856-d73268fac0ac', 'volume_name': 'volume-444d77c2-4003-4c0b-a856-d73268fac0ac', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:08.658347 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (96235d21-ceb3-4107-b8fc-e768e5d0dc79) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:08.662767 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (96235d21-ceb3-4107-b8fc-e768e5d0dc79) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:08.678511 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (30c78ecf-f3a4-46c6-ab6a-10066c33aaee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:08.679697 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Volume 444d77c2-4003-4c0b-a856-d73268fac0ac: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-444d77c2-4003-4c0b-a856-d73268fac0ac', 'volume_size': 1} May 14 21:22:08.680096 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] creating volume 'volume-444d77c2-4003-4c0b-a856-d73268fac0ac' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:22:08.680773 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:09.891928 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:22:09.892597 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:22:09.897669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (30c78ecf-f3a4-46c6-ab6a-10066c33aaee) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-444d77c2-4003-4c0b-a856-d73268fac0ac', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:09.904510 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3d236a3e-eded-4076-9573-7a17b7cc53b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:10.063529 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Volume volume-444d77c2-4003-4c0b-a856-d73268fac0ac (444d77c2-4003-4c0b-a856-d73268fac0ac): created successfully May 14 21:22:10.070551 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3d236a3e-eded-4076-9573-7a17b7cc53b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:10.084516 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Flow 'volume_create_manager' (b17bc378-38e1-413b-9297-1e7d9bd015c4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:10.207046 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-941b8e04-8a0b-47df-8d22-c6840834a002 tempest-VolumesAdminNegativeTest-2091123030 None] Created volume successfully. May 14 21:22:10.767722 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] Lock "b'cinder-444d77c2-4003-4c0b-a856-d73268fac0ac-delete_volume'" acquired by "delete_volume" :: waited 0.025s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:22:10.961447 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:11.206826 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:22:11.207892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] Volume volume-444d77c2-4003-4c0b-a856-d73268fac0ac is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:22:11.334836 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] deleting rbd volume volume-444d77c2-4003-4c0b-a856-d73268fac0ac {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:22:18.476584 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] Created reservations ['8cb1fa3f-200f-43c2-a1b9-f658038f7d37', '94800e63-4f92-4e08-98b1-9cbfb170dbbb', '57b1a043-1e81-4914-88cf-28f8da507b81', '568e4792-904e-42b4-9d71-a0b12a47ed71'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:22:18.637498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:18.672818 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:18.698711 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:22:18.703212 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] Deleted volume successfully. May 14 21:22:18.802195 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-dbdfcde7-f662-40f1-9847-4dae9c220656 tempest-VolumesAdminNegativeTest-2091123030 None] Lock "b'cinder-444d77c2-4003-4c0b-a856-d73268fac0ac-delete_volume'" released by "delete_volume" :: held 8.034s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:22:19.667081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] Lock "b'cinder-2aecd011-8bb0-4a67-b8ce-def64dd1d427-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:22:19.768548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:20.001614 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:22:20.002147 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] Volume volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:22:20.116624 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] deleting rbd volume volume-2aecd011-8bb0-4a67-b8ce-def64dd1d427 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:22:25.983313 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-d53658a4-21b7-4fdc-ab70-edbc1196521e tempest-TaggedBootDevicesTest_v242-387890351 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:22:26.875513 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-d53658a4-21b7-4fdc-ab70-edbc1196521e tempest-TaggedBootDevicesTest_v242-387890351 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.891s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:22:26.876835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-d53658a4-21b7-4fdc-ab70-edbc1196521e tempest-TaggedBootDevicesTest_v242-387890351 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-40376eea-0c0f-488c-ae00-97b792b66c72', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '40376eea-0c0f-488c-ae00-97b792b66c72', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:22:26.979528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-d53658a4-21b7-4fdc-ab70-edbc1196521e tempest-TaggedBootDevicesTest_v242-387890351 None] Connection info returned from driver {'name': 'volumes/volume-40376eea-0c0f-488c-ae00-97b792b66c72', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '40376eea-0c0f-488c-ae00-97b792b66c72', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '808673fc-7787-4534-b341-aa06336bb3ae'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:22:27.107785 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-d53658a4-21b7-4fdc-ab70-edbc1196521e tempest-TaggedBootDevicesTest_v242-387890351 None] attachment_update completed successfully. May 14 21:22:27.424391 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] Created reservations ['73ff2c20-1385-4edc-a179-9a3e863d8197', '5c0734a3-150c-4e11-931a-4cc6d3417e9a', 'b5dd6669-e6bb-49d9-bf96-ca7a5bd75fd1', 'b400bbf4-6bbb-478c-87b4-e355363c97e2'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:22:27.676964 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:27.764501 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:27.847631 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:22:27.852854 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] Deleted volume successfully. May 14 21:22:27.987878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-65e9f3ee-513c-45ec-b123-4ccce272c86f tempest-VolumesAdminNegativeTest-2091123030 None] Lock "b'cinder-2aecd011-8bb0-4a67-b8ce-def64dd1d427-delete_volume'" released by "delete_volume" :: held 8.321s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:22:28.004019 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-64f52fa3-522d-4271-a967-2076c1197d5c tempest-TaggedBootDevicesTest_v242-387890351 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:22:28.929762 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-64f52fa3-522d-4271-a967-2076c1197d5c tempest-TaggedBootDevicesTest_v242-387890351 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.926s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:22:28.938987 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-64f52fa3-522d-4271-a967-2076c1197d5c tempest-TaggedBootDevicesTest_v242-387890351 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-5ab5558f-588e-4b04-84f6-6f28c0048e77', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '5ab5558f-588e-4b04-84f6-6f28c0048e77', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:22:29.076553 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-64f52fa3-522d-4271-a967-2076c1197d5c tempest-TaggedBootDevicesTest_v242-387890351 None] Connection info returned from driver {'name': 'volumes/volume-5ab5558f-588e-4b04-84f6-6f28c0048e77', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '5ab5558f-588e-4b04-84f6-6f28c0048e77', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd4a41f9c-417b-4845-bc6e-9b99f08871bf'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:22:29.179350 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:22:29.221531 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Flow 'volume_create_manager' (8615520a-b9c0-4509-a10d-52d07e0da553) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:29.235852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (868216ea-ae4e-4ce9-ba23-d67a7c251c10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:29.276350 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (868216ea-ae4e-4ce9-ba23-d67a7c251c10) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:22:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachSCSIVolumeTestJSON-volume-846357497',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=66dac191-38b3-4bbd-bf9b-b7ff4ffce383,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='44c96286063c40ac948610903a50e98f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:22:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:22:29Z,use_quota=True,user_id='f2ef9ae6c6d24ba0b85fd8f41afb9c05',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:29.284380 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f6f78922-cecd-42d4-ba5b-5b7ac9d6a26b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:29.290065 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f6f78922-cecd-42d4-ba5b-5b7ac9d6a26b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:29.297044 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (40b054fd-8968-4735-b894-2310ae1d31c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:29.301900 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (40b054fd-8968-4735-b894-2310ae1d31c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'volume_name': 'volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:29.307156 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d2538cce-bb97-4160-a17e-933acd1a9d01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:29.311577 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d2538cce-bb97-4160-a17e-933acd1a9d01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:29.317040 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (579f8f02-f72a-46d6-a4c7-d2e5e29df545) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:29.318405 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Volume 66dac191-38b3-4bbd-bf9b-b7ff4ffce383: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'volume_size': 1} May 14 21:22:29.319034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] creating volume 'volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:22:29.320291 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:29.402520 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-64f52fa3-522d-4271-a967-2076c1197d5c tempest-TaggedBootDevicesTest_v242-387890351 None] attachment_update completed successfully. May 14 21:22:30.325313 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:22:30.325891 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:22:30.331202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (579f8f02-f72a-46d6-a4c7-d2e5e29df545) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:30.336355 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2dc2613e-2854-4742-a711-6d36e4d7edbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:22:30.424628 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Volume volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383 (66dac191-38b3-4bbd-bf9b-b7ff4ffce383): created successfully May 14 21:22:30.428387 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2dc2613e-2854-4742-a711-6d36e4d7edbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:22:30.433923 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Flow 'volume_create_manager' (8615520a-b9c0-4509-a10d-52d07e0da553) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:22:30.556109 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6edcf9d3-6857-4350-9605-34232e44ac9b tempest-AttachSCSIVolumeTestJSON-1113285562 None] Created volume successfully. May 14 21:22:31.179849 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-8c1d9e43-b084-464e-8c19-0fa8dcebdece tempest-TaggedBootDevicesTest_v242-387890351 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:22:32.128843 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-8c1d9e43-b084-464e-8c19-0fa8dcebdece tempest-TaggedBootDevicesTest_v242-387890351 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.948s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:22:32.129610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-8c1d9e43-b084-464e-8c19-0fa8dcebdece tempest-TaggedBootDevicesTest_v242-387890351 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:22:32.338970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-8c1d9e43-b084-464e-8c19-0fa8dcebdece tempest-TaggedBootDevicesTest_v242-387890351 None] Connection info returned from driver {'name': 'volumes/volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '46eb12dc-9f22-43a4-9b76-684627940fdf'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:22:32.498010 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-886cba2f-58ee-4332-81eb-a2e7cbcd23f6 req-8c1d9e43-b084-464e-8c19-0fa8dcebdece tempest-TaggedBootDevicesTest_v242-387890351 None] attachment_update completed successfully. May 14 21:22:32.594238 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-fc877049-1625-40b2-a3e5-3beba6394d09 req-706d37fb-8b3a-4dc1-af4c-91233cec9cc5 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:22:33.812866 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-fc877049-1625-40b2-a3e5-3beba6394d09 req-706d37fb-8b3a-4dc1-af4c-91233cec9cc5 tempest-AttachSCSIVolumeTestJSON-1113285562 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.217s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:22:33.814121 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-fc877049-1625-40b2-a3e5-3beba6394d09 req-706d37fb-8b3a-4dc1-af4c-91233cec9cc5 tempest-AttachSCSIVolumeTestJSON-1113285562 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:22:33.962376 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-fc877049-1625-40b2-a3e5-3beba6394d09 req-706d37fb-8b3a-4dc1-af4c-91233cec9cc5 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Connection info returned from driver {'name': 'volumes/volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '66dac191-38b3-4bbd-bf9b-b7ff4ffce383', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e6ff589a-fda7-486e-879b-7f3ee12f5874'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:22:34.141725 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-fc877049-1625-40b2-a3e5-3beba6394d09 req-706d37fb-8b3a-4dc1-af4c-91233cec9cc5 tempest-AttachSCSIVolumeTestJSON-1113285562 None] attachment_update completed successfully. May 14 21:22:38.615051 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-810e4233-9564-405d-bd8c-9f8d0117e692 req-90c7e239-10db-41ae-aa39-92c84c66a814 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Terminate volume connection completed successfully. May 14 21:22:38.615051 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-810e4233-9564-405d-bd8c-9f8d0117e692 req-90c7e239-10db-41ae-aa39-92c84c66a814 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Deleting attachment e6ff589a-fda7-486e-879b-7f3ee12f5874. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:22:55.061503 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Lock "b'cinder-66dac191-38b3-4bbd-bf9b-b7ff4ffce383-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:22:55.221243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:22:55.491243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:22:55.491728 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Volume volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:22:55.704962 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] deleting rbd volume volume-66dac191-38b3-4bbd-bf9b-b7ff4ffce383 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:23:02.477259 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:23:02.477977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:02.533718 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:02.600099 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:23:03.911880 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Created reservations ['1557a198-4f78-4ce8-a404-d0d720b49189', '8c492546-d6d8-400f-a4fc-e7acc5a1d070', '764437ac-2957-4468-9acc-d70382733884', '2ba8f524-d9d9-4662-80ca-372d51b14ca7'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:23:04.157035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:04.187851 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:04.237208 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:23:04.241889 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Deleted volume successfully. May 14 21:23:04.370009 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5e44bbdb-d573-4fc7-9f90-1a67d69f5d26 tempest-AttachSCSIVolumeTestJSON-1113285562 None] Lock "b'cinder-66dac191-38b3-4bbd-bf9b-b7ff4ffce383-delete_volume'" released by "delete_volume" :: held 9.311s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:23:37.215003 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] Lock "b'cinder-962a3525-29b8-4d27-b336-6be335779521-delete_volume'" acquired by "delete_volume" :: waited 0.016s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:23:37.391745 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:37.673954 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:23:37.674554 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] Volume volume-962a3525-29b8-4d27-b336-6be335779521 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:23:38.786091 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] deleting rbd volume volume-962a3525-29b8-4d27-b336-6be335779521 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:23:48.092446 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] Created reservations ['d1da2e8f-3854-4c72-a085-dd6305932fec', '880a6683-903d-47d7-99ad-f3244ac95198', '0d86ad7b-3d53-48cf-95d9-2e11a8e037c7', '6922248c-9b63-4fa0-bc6a-70df7bb980b3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:23:48.431156 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:48.522625 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:23:48.609496 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:23:48.620294 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] Deleted volume successfully. May 14 21:23:48.769908 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-55290d42-d4b8-47be-aff9-d72123501bb7 tempest-DeleteServersTestJSON-1962247194 None] Lock "b'cinder-962a3525-29b8-4d27-b336-6be335779521-delete_volume'" released by "delete_volume" :: held 11.555s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:24:02.480373 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:24:02.483278 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:24:02.554680 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:24:02.630400 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:24:23.990126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:24:24.022209 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Flow 'volume_create_manager' (3b507e73-5492-4d62-9500-f196b0da47cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:24:24.042351 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d5c07758-54e6-4ebf-97ad-8b69288dddef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:24.076169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d5c07758-54e6-4ebf-97ad-8b69288dddef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:24:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-2140897382',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4b223bea-c1a4-4957-a86b-0fbcf052cae4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fe513cd0819f40d7866ce2aaa1b8ea8d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:24:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:24:24Z,use_quota=True,user_id='87538f2309714545bd47f228b6505883',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:24.086695 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (53e15d52-2d50-4536-842b-bc07ae021657) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:24.099155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (53e15d52-2d50-4536-842b-bc07ae021657) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:24.109672 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ffd11a79-674d-41ac-8865-a7e40fe08d3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:24.121684 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ffd11a79-674d-41ac-8865-a7e40fe08d3f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4b223bea-c1a4-4957-a86b-0fbcf052cae4', 'volume_name': 'volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:24.132123 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d6eec362-10b2-4b04-ad6c-cac98cec5f44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:24.153757 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d6eec362-10b2-4b04-ad6c-cac98cec5f44) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:24.158721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (46aaeddc-d61e-43c9-9a5c-a7af2c88eaa3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:24.165564 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Volume 4b223bea-c1a4-4957-a86b-0fbcf052cae4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4', 'volume_size': 1} May 14 21:24:24.165979 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] creating volume 'volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:24:24.167424 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:24:25.312321 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:24:25.312900 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:24:25.316636 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (46aaeddc-d61e-43c9-9a5c-a7af2c88eaa3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:25.321305 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e691a47f-64c6-4fc7-aa20-420800c6f694) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:25.429446 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Volume volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4 (4b223bea-c1a4-4957-a86b-0fbcf052cae4): created successfully May 14 21:24:25.439632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e691a47f-64c6-4fc7-aa20-420800c6f694) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:25.449883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Flow 'volume_create_manager' (3b507e73-5492-4d62-9500-f196b0da47cb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:24:25.525658 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a3a53d62-7bc6-4a95-9cec-07228ac936ef tempest-ServersNegativeTestJSON-663937628 None] Created volume successfully. May 14 21:24:28.156155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:24:28.189624 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Flow 'volume_create_manager' (364fd3ba-785d-4ab7-a97d-e783f482f32c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:24:28.196416 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6de479c8-5d1a-4174-b51b-19ac844fb5f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:28.217992 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6de479c8-5d1a-4174-b51b-19ac844fb5f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:24:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-1998134898',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6f8096aa-7bba-4f4f-97a3-7aab39d69227,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fe513cd0819f40d7866ce2aaa1b8ea8d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:24:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:24:28Z,use_quota=True,user_id='87538f2309714545bd47f228b6505883',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:28.223690 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5ca24cd0-147a-42fc-a946-4fb4cc6b5b1a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:28.227870 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5ca24cd0-147a-42fc-a946-4fb4cc6b5b1a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:28.233576 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (815be3bf-9e58-4c61-93af-f3f663479695) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:28.238548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (815be3bf-9e58-4c61-93af-f3f663479695) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6f8096aa-7bba-4f4f-97a3-7aab39d69227', 'volume_name': 'volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:28.245737 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (273ac19b-8a12-4f39-b030-bb7ad68c2c4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:28.250096 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (273ac19b-8a12-4f39-b030-bb7ad68c2c4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:28.256379 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4f0f8f68-2e82-4397-9b4d-51678501c8a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:28.257527 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Volume 6f8096aa-7bba-4f4f-97a3-7aab39d69227: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227', 'volume_size': 1} May 14 21:24:28.257923 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] creating volume 'volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:24:28.258607 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:24:29.489061 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:24:29.490382 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:24:29.495498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4f0f8f68-2e82-4397-9b4d-51678501c8a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:29.500477 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd82989c-2912-402c-a6c0-fac314800792) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:24:29.555182 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Volume volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227 (6f8096aa-7bba-4f4f-97a3-7aab39d69227): created successfully May 14 21:24:29.560637 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd82989c-2912-402c-a6c0-fac314800792) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:24:29.568166 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Flow 'volume_create_manager' (364fd3ba-785d-4ab7-a97d-e783f482f32c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:24:29.632117 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2b80fda4-98ce-45d4-8391-f1d1ad04f6e3 tempest-ServersNegativeTestJSON-663937628 None] Created volume successfully. May 14 21:25:02.486084 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:25:02.487508 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:25:02.543169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:25:02.632628 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:26:02.490497 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:26:02.491667 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:02.529613 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:02.602009 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:26:21.574132 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-4dd2e3b6-0d23-4bad-9233-ce02cccc58fc tempest-ServersTestBootFromVolume-823166955 None] Terminate volume connection completed successfully. May 14 21:26:21.575225 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-4dd2e3b6-0d23-4bad-9233-ce02cccc58fc tempest-ServersTestBootFromVolume-823166955 None] Deleting attachment ed42ed9b-ba24-462c-bad2-ac06362b22ee. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:26:22.218236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] Lock "b'cinder-022c11da-58a1-4855-977c-de51c3870d3f-delete_volume'" acquired by "delete_volume" :: waited 0.022s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:26:22.378673 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:22.677197 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:26:23.075184 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] deleting rbd volume volume-022c11da-58a1-4855-977c-de51c3870d3f {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:26:30.151977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] Lock "b'cinder-6f8096aa-7bba-4f4f-97a3-7aab39d69227-delete_volume'" acquired by "delete_volume" :: waited 0.041s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:26:30.409882 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:30.802825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:26:30.803985 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] Volume volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:26:31.030060 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] deleting rbd volume volume-6f8096aa-7bba-4f4f-97a3-7aab39d69227 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:26:32.445284 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:26:32.714263 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] deleting parent snapshot volume-022c11da-58a1-4855-977c-de51c3870d3f.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:26:36.935103 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] Created reservations ['dd45b0a8-f47f-4c6a-af35-c717d1fe34e8', '859d9732-ed6a-41b8-9735-de8d49696d93', 'd22dd80a-6834-4488-9169-8ea777fce87a', 'c2b68b37-8a57-4d79-8338-a1429af41a25'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:26:37.165083 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:37.312896 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:37.369815 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:26:37.382358 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] Deleted volume successfully. May 14 21:26:37.466056 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-7210e662-f1d3-4a60-9dbf-e0429b3890bf req-6252f20b-4460-4ce1-9e65-ed58984dffda tempest-ServersTestBootFromVolume-823166955 None] Lock "b'cinder-022c11da-58a1-4855-977c-de51c3870d3f-delete_volume'" released by "delete_volume" :: held 15.242s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:26:38.289335 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] Created reservations ['ea3a7635-1117-4206-b4dd-76f19cea8f5a', '9b6422f0-c40f-43c3-8c6c-bdd77a2b7419', 'e764b91d-ba21-47bd-85f8-2631073cd74c', '0168c123-5eb0-4326-81d5-ff685e7dc893'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:26:38.454977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:38.508586 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:38.574140 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:26:38.578777 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] Deleted volume successfully. May 14 21:26:38.679135 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0f7742ed-16d9-41eb-9da4-f559846fcc63 tempest-ServersNegativeTestJSON-663937628 None] Lock "b'cinder-6f8096aa-7bba-4f4f-97a3-7aab39d69227-delete_volume'" released by "delete_volume" :: held 8.527s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:26:39.068373 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] Lock "b'cinder-4b223bea-c1a4-4957-a86b-0fbcf052cae4-delete_volume'" acquired by "delete_volume" :: waited 0.016s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:26:39.194827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:39.551189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:26:39.552065 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] Volume volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:26:39.803471 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] deleting rbd volume volume-4b223bea-c1a4-4957-a86b-0fbcf052cae4 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:26:46.511713 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] Created reservations ['d7d908a9-4de4-402e-9033-8abc4a250c52', '007046bf-f333-43c2-85ba-87d1020eb8b0', '6c923bd8-8bd5-4a82-a5dd-813244229d65', '0db86bf1-d336-4991-814b-dc0084f565b1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:26:46.690226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:46.797582 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:26:46.913613 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:26:46.919711 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] Deleted volume successfully. May 14 21:26:46.999340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6b6a011e-1adb-4ea0-af5e-31ba59dd17ef tempest-ServersNegativeTestJSON-663937628 None] Lock "b'cinder-4b223bea-c1a4-4957-a86b-0fbcf052cae4-delete_volume'" released by "delete_volume" :: held 7.928s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:27:00.365171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:00.397926 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Flow 'volume_create_manager' (4e27ea1b-b7ee-4a6c-93cb-e5b575e6d2cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:00.406495 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6951ffe-6106-4487-9770-3beef33081eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:00.432799 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6951ffe-6106-4487-9770-3beef33081eb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:26:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-471871734',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9f2747e4-5e82-4cee-920f-2d2268ac18bb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8bf1c9f99ca0403f9b22031eb90ffe8f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:00Z,use_quota=True,user_id='1382428c1e50441794230e9abc5f2d39',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:00.440200 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c3fd387a-df59-41cb-a27c-06bb08148e47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:00.445791 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c3fd387a-df59-41cb-a27c-06bb08148e47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:00.453636 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (90147747-bec6-4458-a47b-a43bf29dc75a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:00.460351 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (90147747-bec6-4458-a47b-a43bf29dc75a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9f2747e4-5e82-4cee-920f-2d2268ac18bb', 'volume_name': 'volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:00.466385 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9be3cbd6-b4a5-4ad2-8692-64f45b30cf49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:00.471124 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9be3cbd6-b4a5-4ad2-8692-64f45b30cf49) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:00.476774 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (97aa2223-7cf5-4958-8499-cb5301965aab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:00.478251 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Volume 9f2747e4-5e82-4cee-920f-2d2268ac18bb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb', 'volume_size': 1} May 14 21:27:00.478834 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] creating volume 'volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:27:00.479869 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:01.766402 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:01.767507 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:01.772494 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (97aa2223-7cf5-4958-8499-cb5301965aab) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:01.782039 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (75ae4ad2-c824-494b-8fc6-3c107d5c0ef9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:01.885711 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Volume volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb (9f2747e4-5e82-4cee-920f-2d2268ac18bb): created successfully May 14 21:27:01.885711 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (75ae4ad2-c824-494b-8fc6-3c107d5c0ef9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:01.891530 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Flow 'volume_create_manager' (4e27ea1b-b7ee-4a6c-93cb-e5b575e6d2cf) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:02.012624 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d3832cbc-7dc3-42ab-bd06-7f0a2bd537bc tempest-VolumesTransfersV355Test-1888999586 None] Created volume successfully. May 14 21:27:02.493018 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:27:02.494037 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:02.579093 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:02.646574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:27:03.330843 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:03.365983 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Flow 'volume_create_manager' (437a696b-0e53-43de-a643-73e9d3fecf3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:03.371663 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (728d462a-b357-467d-bfca-4e58f3c38c87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:03.397054 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (728d462a-b357-467d-bfca-4e58f3c38c87) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-677096663',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=252f69af-c210-4b9b-8b32-e9dc857c2b60,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1df1ae61bf82472dadc4830840421a63',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:03Z,use_quota=True,user_id='22a06bece449427d91289ade79c94822',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:03.402705 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d5bfb78d-335b-4171-938d-3ecf397e10bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:03.408427 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d5bfb78d-335b-4171-938d-3ecf397e10bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:03.414976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3acddb3d-d57b-4dda-8c20-fceb93805f11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:03.604002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3acddb3d-d57b-4dda-8c20-fceb93805f11) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '252f69af-c210-4b9b-8b32-e9dc857c2b60', 'volume_name': 'volume-252f69af-c210-4b9b-8b32-e9dc857c2b60', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:03.620196 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cea79420-5935-48b9-baa8-4cbe2012498c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:03.624236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cea79420-5935-48b9-baa8-4cbe2012498c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:03.630306 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6a8cf91d-40c7-42e8-9cf9-50b6d22563be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:03.631650 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Volume 252f69af-c210-4b9b-8b32-e9dc857c2b60: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-252f69af-c210-4b9b-8b32-e9dc857c2b60', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:27:03.640953 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Cloning 252f69af-c210-4b9b-8b32-e9dc857c2b60 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:27:03.669597 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.032s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:27:03.757476 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 27, 3, 676408)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:27:03.759575 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:27:03.760340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:27:03.774375 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.109s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:27:03.775541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:27:03.790781 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 27, 3, 781531)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:27:03.791604 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:27:03.792231 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:27:03.810200 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:04.390775 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] creating snapshot='volume-252f69af-c210-4b9b-8b32-e9dc857c2b60.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:27:07.724236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-252f69af-c210-4b9b-8b32-e9dc857c2b60.clone_snap' to 'volume-252f69af-c210-4b9b-8b32-e9dc857c2b60' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:27:07.822081 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d3870940-3383-457a-b693-5bd865f305af tempest-VolumesTransfersV355Test-971912654 None] Transfer volume completed successfully. May 14 21:27:07.979614 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:08.027465 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Flow 'volume_create_manager' (79ba9df4-5cdc-4f58-afe7-73d4fbf677fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:08.076284 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (350a4d1b-efa3-48d1-850b-86bad1d01998) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:08.134680 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (350a4d1b-efa3-48d1-850b-86bad1d01998) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1780949872',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1ba2acd5-df14-446f-a4d1-701795380c04,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8bd6f25048cd4798985e725f734f1845',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:04Z,use_quota=True,user_id='1e3390c9579a4f598c46f9417b4dd046',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:08.166454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (47839862-3d4c-41b5-9591-2c4e3326ed6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:08.184133 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (47839862-3d4c-41b5-9591-2c4e3326ed6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:08.202209 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d7b5fe94-f2a8-42d3-97ea-78d5c8901acb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:08.220035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d7b5fe94-f2a8-42d3-97ea-78d5c8901acb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1ba2acd5-df14-446f-a4d1-701795380c04', 'volume_name': 'volume-1ba2acd5-df14-446f-a4d1-701795380c04', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:08.230494 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (be79839a-372b-4b6c-afa6-b3789f5dbf58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:08.240828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (be79839a-372b-4b6c-afa6-b3789f5dbf58) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:08.260914 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7c960432-2d86-433d-860a-3435ac058463) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:08.264384 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Volume 1ba2acd5-df14-446f-a4d1-701795380c04: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1ba2acd5-df14-446f-a4d1-701795380c04', 'volume_size': 1} May 14 21:27:08.267608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] creating volume 'volume-1ba2acd5-df14-446f-a4d1-701795380c04' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:27:08.269082 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:09.796337 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] Lock "b'cinder-9f2747e4-5e82-4cee-920f-2d2268ac18bb-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:27:09.880877 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:09.881871 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:09.892121 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7c960432-2d86-433d-860a-3435ac058463) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1ba2acd5-df14-446f-a4d1-701795380c04', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:09.895204 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a544b22-9eb9-4469-a1bd-559fa3dd91d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:09.938254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:10.157035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:27:10.157791 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] Volume volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:27:10.443748 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] deleting rbd volume volume-9f2747e4-5e82-4cee-920f-2d2268ac18bb {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:27:10.521216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:27:10.525291 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:10.525863 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:11.272787 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:27:11.277065 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Marking volume 252f69af-c210-4b9b-8b32-e9dc857c2b60 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:27:11.280418 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Volume volume-1ba2acd5-df14-446f-a4d1-701795380c04 (1ba2acd5-df14-446f-a4d1-701795380c04): created successfully May 14 21:27:11.284547 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a544b22-9eb9-4469-a1bd-559fa3dd91d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:11.304513 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Flow 'volume_create_manager' (79ba9df4-5cdc-4f58-afe7-73d4fbf677fe) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:11.425759 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 252f69af-c210-4b9b-8b32-e9dc857c2b60. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:27:11.427030 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Creating volume glance metadata for volume 252f69af-c210-4b9b-8b32-e9dc857c2b60 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:27:11.466972 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-38e5b0aa-6253-4fb4-8578-3ee3bf5bd9c6 tempest-AttachVolumeNegativeTest-1286916119 None] Created volume successfully. May 14 21:27:11.664309 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6a8cf91d-40c7-42e8-9cf9-50b6d22563be) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-252f69af-c210-4b9b-8b32-e9dc857c2b60', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:11.677496 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1a0f38f6-637d-41f2-8fc1-805c479d5752) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:11.799046 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Volume volume-252f69af-c210-4b9b-8b32-e9dc857c2b60 (252f69af-c210-4b9b-8b32-e9dc857c2b60): created successfully May 14 21:27:11.810714 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1a0f38f6-637d-41f2-8fc1-805c479d5752) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:11.826857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Flow 'volume_create_manager' (437a696b-0e53-43de-a643-73e9d3fecf3d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:11.905730 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4c2bfe9d-8360-49bc-83ba-fb09ced1fe83 tempest-TaggedBootDevicesTest-1853898337 None] Created volume successfully. May 14 21:27:12.979593 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-4b8411ae-9ce5-4607-88f6-5fd96a25a414 req-dabbd7ac-e498-46a1-adbb-192e5d29529d tempest-AttachVolumeNegativeTest-1286916119 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:27:13.456195 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:13.548990 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Flow 'volume_create_manager' (8f514be1-9b3a-44b7-8815-e491817cf8eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:13.567798 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (898cbd5d-32e0-4ca6-b975-76462d1decec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:13.614230 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (898cbd5d-32e0-4ca6-b975-76462d1decec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-132773421',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=12a0e3c5-2efb-4eb6-a302-15e9e7d28509,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1df1ae61bf82472dadc4830840421a63',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:13Z,use_quota=True,user_id='22a06bece449427d91289ade79c94822',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:13.626773 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9f4ad5e9-9d91-46e7-9b41-336112c6a6d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:13.637969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9f4ad5e9-9d91-46e7-9b41-336112c6a6d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:13.654581 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (91b0b335-59b0-483b-944f-8864fbb717cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:13.670404 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (91b0b335-59b0-483b-944f-8864fbb717cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'volume_name': 'volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:13.688488 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3cd44236-4093-4ae8-b404-f0f091830410) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:13.699041 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3cd44236-4093-4ae8-b404-f0f091830410) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:13.705031 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (775d9bc7-9df0-4828-9af2-8c33d4add588) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:13.712249 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Volume 12a0e3c5-2efb-4eb6-a302-15e9e7d28509: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'volume_size': 1} May 14 21:27:13.713879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] creating volume 'volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:27:13.715016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:14.307071 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-4b8411ae-9ce5-4607-88f6-5fd96a25a414 req-dabbd7ac-e498-46a1-adbb-192e5d29529d tempest-AttachVolumeNegativeTest-1286916119 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.328s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:27:14.308725 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-4b8411ae-9ce5-4607-88f6-5fd96a25a414 req-dabbd7ac-e498-46a1-adbb-192e5d29529d tempest-AttachVolumeNegativeTest-1286916119 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1ba2acd5-df14-446f-a4d1-701795380c04', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '1ba2acd5-df14-446f-a4d1-701795380c04', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:27:14.425701 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-4b8411ae-9ce5-4607-88f6-5fd96a25a414 req-dabbd7ac-e498-46a1-adbb-192e5d29529d tempest-AttachVolumeNegativeTest-1286916119 None] Connection info returned from driver {'name': 'volumes/volume-1ba2acd5-df14-446f-a4d1-701795380c04', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1ba2acd5-df14-446f-a4d1-701795380c04', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a362418e-f1c7-4b9c-ba9e-e361e3e6cddf'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:27:14.722760 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-4b8411ae-9ce5-4607-88f6-5fd96a25a414 req-dabbd7ac-e498-46a1-adbb-192e5d29529d tempest-AttachVolumeNegativeTest-1286916119 None] attachment_update completed successfully. May 14 21:27:15.035890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:15.036427 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:15.089995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (775d9bc7-9df0-4828-9af2-8c33d4add588) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:15.102464 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dc359bd3-fcc8-49b4-86d9-285ccca04a38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:15.208529 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Volume volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509 (12a0e3c5-2efb-4eb6-a302-15e9e7d28509): created successfully May 14 21:27:15.212254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dc359bd3-fcc8-49b4-86d9-285ccca04a38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:15.217832 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Flow 'volume_create_manager' (8f514be1-9b3a-44b7-8815-e491817cf8eb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:15.334482 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-21a47796-bac9-4e96-8326-1e29c8655961 tempest-TaggedBootDevicesTest-1853898337 None] Created volume successfully. May 14 21:27:17.597984 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:17.654695 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Flow 'volume_create_manager' (dcaa58e2-9c59-42d2-aeff-f1b69da30845) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:17.670930 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5b615a97-b872-423a-8eec-361c72820d66) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:17.693563 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5b615a97-b872-423a-8eec-361c72820d66) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-851488171',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7d2d29a8-c865-4570-a305-7473b65a3efb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1df1ae61bf82472dadc4830840421a63',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:17Z,use_quota=True,user_id='22a06bece449427d91289ade79c94822',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:17.699929 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (043417ae-3daf-4eca-9794-e21b0cae4597) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:17.704490 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (043417ae-3daf-4eca-9794-e21b0cae4597) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:17.710860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ea1ebae-f831-4909-801d-9a789a3c8d93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:17.716534 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ea1ebae-f831-4909-801d-9a789a3c8d93) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7d2d29a8-c865-4570-a305-7473b65a3efb', 'volume_name': 'volume-7d2d29a8-c865-4570-a305-7473b65a3efb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:17.724482 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cf0bd0ab-b8bb-4a98-9f20-4857325d6104) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:17.729806 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cf0bd0ab-b8bb-4a98-9f20-4857325d6104) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:17.734996 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1eb32bd9-1750-4ea6-91a8-cb746ff880d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:17.736447 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Volume 7d2d29a8-c865-4570-a305-7473b65a3efb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7d2d29a8-c865-4570-a305-7473b65a3efb', 'volume_size': 1} May 14 21:27:17.737093 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] creating volume 'volume-7d2d29a8-c865-4570-a305-7473b65a3efb' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:27:17.738017 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:18.486334 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] Created reservations ['c4957ee2-a65f-4bd6-86fd-1fbdd7d5ac8b', 'c9dc93c1-e53d-46eb-b920-88e4698a7e81', '0854fa6f-b34e-4bf4-bce6-f327f4a791ef', '5fee24b7-8787-4337-8502-c98b8702dbcb'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:27:18.703774 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:18.791344 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:18.852039 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:27:18.856701 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] Deleted volume successfully. May 14 21:27:18.889971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:18.890716 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:18.897561 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1eb32bd9-1750-4ea6-91a8-cb746ff880d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7d2d29a8-c865-4570-a305-7473b65a3efb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:18.904436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (35e7dad6-25f5-4145-ab3d-2cb816af2401) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:19.063577 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Volume volume-7d2d29a8-c865-4570-a305-7473b65a3efb (7d2d29a8-c865-4570-a305-7473b65a3efb): created successfully May 14 21:27:19.076427 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (35e7dad6-25f5-4145-ab3d-2cb816af2401) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:19.088486 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Flow 'volume_create_manager' (dcaa58e2-9c59-42d2-aeff-f1b69da30845) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:19.097346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8cf066ac-4a91-44de-9d9a-bed7ad4e1503 tempest-VolumesTransfersV355Test-790625186 None] Lock "b'cinder-9f2747e4-5e82-4cee-920f-2d2268ac18bb-delete_volume'" released by "delete_volume" :: held 9.301s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:27:19.219156 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e3ec1f41-9dca-4041-ad71-c77be6e80abc tempest-TaggedBootDevicesTest-1853898337 None] Created volume successfully. May 14 21:27:20.690265 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:20.735375 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Flow 'volume_create_manager' (df4ab2f7-d29d-4a08-9a5b-2f00722ba96a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:20.756685 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (037950a5-b5d0-4208-bbe6-e0a5400dd022) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:20.775385 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (037950a5-b5d0-4208-bbe6-e0a5400dd022) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-381334813',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5bac2855-6bb6-40c8-8c46-ffd33ec43f7d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8bf1c9f99ca0403f9b22031eb90ffe8f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:20Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:20Z,use_quota=True,user_id='1382428c1e50441794230e9abc5f2d39',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:20.781548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb2263bb-5d41-49b4-8b11-e5d99e49817f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:20.787523 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb2263bb-5d41-49b4-8b11-e5d99e49817f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:20.793649 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (47548c5c-3fdf-46f6-bd05-e325607eecbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:20.799753 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (47548c5c-3fdf-46f6-bd05-e325607eecbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5bac2855-6bb6-40c8-8c46-ffd33ec43f7d', 'volume_name': 'volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:20.805377 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (30f02538-b762-48b5-9460-13c583e7a364) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:20.810364 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (30f02538-b762-48b5-9460-13c583e7a364) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:20.815423 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (111748d5-4731-49cb-8001-03ba949ae3fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:20.817819 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Volume 5bac2855-6bb6-40c8-8c46-ffd33ec43f7d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d', 'volume_size': 1} May 14 21:27:20.818419 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] creating volume 'volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:27:20.819407 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:22.068905 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:22.069517 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:22.073254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (111748d5-4731-49cb-8001-03ba949ae3fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:22.082812 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6ef1d588-20de-4b74-a5fa-f9b8e9923bda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:22.191123 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Volume volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d (5bac2855-6bb6-40c8-8c46-ffd33ec43f7d): created successfully May 14 21:27:22.196254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6ef1d588-20de-4b74-a5fa-f9b8e9923bda) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:22.205653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Flow 'volume_create_manager' (df4ab2f7-d29d-4a08-9a5b-2f00722ba96a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:22.294085 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bcc0eef7-fa8f-46f8-8b7c-1031c6910681 tempest-VolumesTransfersV355Test-1888999586 None] Created volume successfully. May 14 21:27:23.427683 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] Lock "b'cinder-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d-delete_volume'" acquired by "delete_volume" :: waited 0.021s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:27:23.608834 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:23.859420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:27:23.862316 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] Volume volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:27:24.277130 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] deleting rbd volume volume-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:27:32.266455 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] Created reservations ['90e03ea0-ce26-4325-a57c-f2091dacf252', '6ac50e1f-1bf2-4d20-b79d-ab426aab64ff', '9b4c8c85-4d95-4d06-86f7-8bcd6f6224ec', '523e45ed-a6d0-43dc-a6c8-d41fec2c28f0'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:27:32.540653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:32.590002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:32.643600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:27:32.648318 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] Deleted volume successfully. May 14 21:27:32.732729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-73673bb3-2179-4682-b303-332b589aeb59 tempest-VolumesTransfersV355Test-790625186 None] Lock "b'cinder-5bac2855-6bb6-40c8-8c46-ffd33ec43f7d-delete_volume'" released by "delete_volume" :: held 9.313s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:27:40.941440 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:40.962580 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (c08056c1-fd83-4ffa-870f-0f55e23c95f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:40.969278 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1313e7c5-1dcb-4fdf-9484-7f2570a1f1bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:40.999511 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1313e7c5-1dcb-4fdf-9484-7f2570a1f1bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-924751278',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ec66fdb7-4204-4ce7-befa-e3acd2c72003,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='51bdb907e2014a05815af94bc5cff92d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:41Z,use_quota=True,user_id='d698e25b4c5d409c8f14c353218baac2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:41.005114 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (38d78856-b75d-49f5-b116-82b8ca376c69) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:41.010285 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (38d78856-b75d-49f5-b116-82b8ca376c69) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:41.016164 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (67beeec1-7d5c-4071-b305-27e53a94cb9c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:41.124526 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (67beeec1-7d5c-4071-b305-27e53a94cb9c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'volume_name': 'volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:41.131179 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34d8a99b-3a55-4141-b872-76f6eeced5ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:41.135536 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34d8a99b-3a55-4141-b872-76f6eeced5ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:41.144310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (38e3f85d-d064-45a6-933a-9b76e82c1402) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:41.144310 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Volume ec66fdb7-4204-4ce7-befa-e3acd2c72003: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:27:41.145160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Cloning ec66fdb7-4204-4ce7-befa-e3acd2c72003 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:27:41.172931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.025s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:27:41.287911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 27, 41, 183718)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:27:41.288716 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:27:41.289339 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:27:41.295055 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.123s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:27:41.296780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:27:41.304837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 27, 41, 301564)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:27:41.305542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:27:41.306183 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:27:41.319533 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:41.710803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] creating snapshot='volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:27:44.056728 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003.clone_snap' to 'volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:27:44.080479 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-1f826788-5d53-48df-9b35-99acb88db49c tempest-TaggedBootDevicesTest-1853898337 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:27:44.882827 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-1f826788-5d53-48df-9b35-99acb88db49c tempest-TaggedBootDevicesTest-1853898337 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.802s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:27:44.884555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-1f826788-5d53-48df-9b35-99acb88db49c tempest-TaggedBootDevicesTest-1853898337 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-252f69af-c210-4b9b-8b32-e9dc857c2b60', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '252f69af-c210-4b9b-8b32-e9dc857c2b60', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:27:45.031039 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-1f826788-5d53-48df-9b35-99acb88db49c tempest-TaggedBootDevicesTest-1853898337 None] Connection info returned from driver {'name': 'volumes/volume-252f69af-c210-4b9b-8b32-e9dc857c2b60', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '252f69af-c210-4b9b-8b32-e9dc857c2b60', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'f3ac3501-aff2-41ff-a7ba-a6b0e9645f89'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:27:45.215791 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-1f826788-5d53-48df-9b35-99acb88db49c tempest-TaggedBootDevicesTest-1853898337 None] attachment_update completed successfully. May 14 21:27:46.182254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:27:46.183464 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:46.183875 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:46.874118 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:27:46.875006 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Marking volume ec66fdb7-4204-4ce7-befa-e3acd2c72003 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:27:46.913275 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-be3edf1e-e60b-475e-be85-8c8651860d8b tempest-TaggedBootDevicesTest-1853898337 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:27:46.963707 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to ec66fdb7-4204-4ce7-befa-e3acd2c72003. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:27:46.964311 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Creating volume glance metadata for volume ec66fdb7-4204-4ce7-befa-e3acd2c72003 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:27:47.194719 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (38e3f85d-d064-45a6-933a-9b76e82c1402) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:47.201226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f7ddde77-6b0b-4402-a826-a01c795c3fb7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:47.281859 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Volume volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003 (ec66fdb7-4204-4ce7-befa-e3acd2c72003): created successfully May 14 21:27:47.288305 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f7ddde77-6b0b-4402-a826-a01c795c3fb7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:47.297784 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (c08056c1-fd83-4ffa-870f-0f55e23c95f9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:47.396157 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3c033486-8c42-4b70-8142-495322b37416 tempest-VolumesCloneTest-218403688 None] Created volume successfully. May 14 21:27:47.948818 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-be3edf1e-e60b-475e-be85-8c8651860d8b tempest-TaggedBootDevicesTest-1853898337 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.036s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:27:47.949449 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-be3edf1e-e60b-475e-be85-8c8651860d8b tempest-TaggedBootDevicesTest-1853898337 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:27:48.048638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-be3edf1e-e60b-475e-be85-8c8651860d8b tempest-TaggedBootDevicesTest-1853898337 None] Connection info returned from driver {'name': 'volumes/volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '12a0e3c5-2efb-4eb6-a302-15e9e7d28509', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5db66234-e93a-44df-84df-a38baa9ead09'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:27:48.302904 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-be3edf1e-e60b-475e-be85-8c8651860d8b tempest-TaggedBootDevicesTest-1853898337 None] attachment_update completed successfully. May 14 21:27:49.209187 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-045ffb5c-28fc-44bd-b6c1-a960c0b9cf39 tempest-TaggedBootDevicesTest-1853898337 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:27:49.626992 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:49.675451 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (182a8dc5-95fd-4dbf-b69a-62b0e8d9c204) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:49.685567 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6d349c20-9c57-4144-b067-a0c4bd8e4f57) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:49.711709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6d349c20-9c57-4144-b067-a0c4bd8e4f57) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-2078180178',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9d841ce2-7872-4287-9cab-5db012a7caee,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='51bdb907e2014a05815af94bc5cff92d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=ec66fdb7-4204-4ce7-befa-e3acd2c72003,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:49Z,use_quota=True,user_id='d698e25b4c5d409c8f14c353218baac2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:49.717999 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (76c5da16-374f-47ad-8ba3-c63b129674da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:49.722425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (76c5da16-374f-47ad-8ba3-c63b129674da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:49.728095 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fe87f50d-018a-4a9b-991f-94b4b5faaab9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:49.749999 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fe87f50d-018a-4a9b-991f-94b4b5faaab9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '9d841ce2-7872-4287-9cab-5db012a7caee', 'volume_name': 'volume-9d841ce2-7872-4287-9cab-5db012a7caee', 'volume_size': 1, 'source_volid': 'ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'source_volstatus': 'available'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:49.756429 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fd541b4a-986d-4f10-b0d1-f6f68dd23785) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:49.760586 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fd541b4a-986d-4f10-b0d1-f6f68dd23785) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:49.765988 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (93bb3e51-8de9-4ecb-8bbe-a8e180d7a8e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:49.767131 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Volume 9d841ce2-7872-4287-9cab-5db012a7caee: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-9d841ce2-7872-4287-9cab-5db012a7caee', 'volume_size': 1, 'source_volid': 'ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'source_volstatus': 'available'} May 14 21:27:49.793203 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:50.041519 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] creating snapshot='volume-9d841ce2-7872-4287-9cab-5db012a7caee.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:27:51.988202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] cloning 'volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003@volume-9d841ce2-7872-4287-9cab-5db012a7caee.clone_snap' to 'volume-9d841ce2-7872-4287-9cab-5db012a7caee' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:27:52.000822 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-045ffb5c-28fc-44bd-b6c1-a960c0b9cf39 tempest-TaggedBootDevicesTest-1853898337 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.791s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:27:52.003098 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-045ffb5c-28fc-44bd-b6c1-a960c0b9cf39 tempest-TaggedBootDevicesTest-1853898337 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-7d2d29a8-c865-4570-a305-7473b65a3efb', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '7d2d29a8-c865-4570-a305-7473b65a3efb', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:27:52.151936 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-045ffb5c-28fc-44bd-b6c1-a960c0b9cf39 tempest-TaggedBootDevicesTest-1853898337 None] Connection info returned from driver {'name': 'volumes/volume-7d2d29a8-c865-4570-a305-7473b65a3efb', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '7d2d29a8-c865-4570-a305-7473b65a3efb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '747510e5-914e-4798-a034-87797f5044d5'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:27:52.321440 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f6d64d2f-110d-4b97-b4ac-b9f5f16b4419 req-045ffb5c-28fc-44bd-b6c1-a960c0b9cf39 tempest-TaggedBootDevicesTest-1853898337 None] attachment_update completed successfully. May 14 21:27:54.435715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:27:54.437433 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:54.438010 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:55.186045 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:27:55.187608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Marking volume 9d841ce2-7872-4287-9cab-5db012a7caee as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:27:55.299261 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Copying metadata from source volume ec66fdb7-4204-4ce7-befa-e3acd2c72003 to 9d841ce2-7872-4287-9cab-5db012a7caee. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:436}} May 14 21:27:55.446100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (93bb3e51-8de9-4ecb-8bbe-a8e180d7a8e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9d841ce2-7872-4287-9cab-5db012a7caee', 'volume_size': 1, 'source_volid': 'ec66fdb7-4204-4ce7-befa-e3acd2c72003', 'source_volstatus': 'available'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:55.451922 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9e7c6a80-5340-4aca-9e8b-6b08b89a457a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:55.556380 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Volume volume-9d841ce2-7872-4287-9cab-5db012a7caee (9d841ce2-7872-4287-9cab-5db012a7caee): created successfully May 14 21:27:55.560419 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9e7c6a80-5340-4aca-9e8b-6b08b89a457a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:55.570198 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (182a8dc5-95fd-4dbf-b69a-62b0e8d9c204) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:55.649520 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-330569a0-4845-47e4-9a54-acd675dee2ea tempest-VolumesCloneTest-218403688 None] Created volume successfully. May 14 21:27:56.815256 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:56.834899 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (2fcd11c1-c648-4f28-9b8f-329bd2efc2f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:56.841343 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (41649055-48ac-4f3a-a822-d4b9bd9c1a66) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:56.859353 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (41649055-48ac-4f3a-a822-d4b9bd9c1a66) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1038749185',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bb5dc462-0184-4114-be06-448971acfe05,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='51bdb907e2014a05815af94bc5cff92d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:27:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:27:57Z,use_quota=True,user_id='d698e25b4c5d409c8f14c353218baac2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:56.864875 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (190af82a-8c30-4a60-81cd-ffb7c463bc8c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:56.868933 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (190af82a-8c30-4a60-81cd-ffb7c463bc8c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:56.873970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ffce32d1-9cc6-472e-964c-f87898fbee8b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:56.878481 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ffce32d1-9cc6-472e-964c-f87898fbee8b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bb5dc462-0184-4114-be06-448971acfe05', 'volume_name': 'volume-bb5dc462-0184-4114-be06-448971acfe05', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:56.883451 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a02b80ab-db74-4c6a-b8d9-9a6801216ce6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:56.887265 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a02b80ab-db74-4c6a-b8d9-9a6801216ce6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:56.892178 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (97b4e5e5-0109-4146-9a98-27be5bce766e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:56.893843 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Volume bb5dc462-0184-4114-be06-448971acfe05: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bb5dc462-0184-4114-be06-448971acfe05', 'volume_size': 1} May 14 21:27:56.894231 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] creating volume 'volume-bb5dc462-0184-4114-be06-448971acfe05' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:27:56.895105 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:27:57.999827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:27:57.999827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:27:57.999827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (97b4e5e5-0109-4146-9a98-27be5bce766e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bb5dc462-0184-4114-be06-448971acfe05', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:58.009762 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a8193b85-ff4d-4040-8d4c-713f6db291be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:27:58.144966 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Volume volume-bb5dc462-0184-4114-be06-448971acfe05 (bb5dc462-0184-4114-be06-448971acfe05): created successfully May 14 21:27:58.153484 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a8193b85-ff4d-4040-8d4c-713f6db291be) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:27:58.185849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (2fcd11c1-c648-4f28-9b8f-329bd2efc2f4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:58.265660 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6c5a9dbf-a8a3-4de2-8550-96c6ee43bab0 tempest-VolumesCloneTest-218403688 None] Created volume successfully. May 14 21:27:59.879191 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:27:59.962194 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (789fd4a3-11bd-471b-b376-77fe9b817f4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:27:59.985943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7b4f1f73-9411-4c2a-8e25-3f3b9318e642) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:00.027288 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7b4f1f73-9411-4c2a-8e25-3f3b9318e642) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:27:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1366419609',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3625ab3c-1eed-48fc-94c0-694f99a884df,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='51bdb907e2014a05815af94bc5cff92d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:28:00Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=bb5dc462-0184-4114-be06-448971acfe05,status='creating',terminated_at=None,updated_at=2022-05-14T21:28:00Z,use_quota=True,user_id='d698e25b4c5d409c8f14c353218baac2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:00.034189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4f8e9cab-77ff-4c27-bb08-cf1ae8455a7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:00.044004 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4f8e9cab-77ff-4c27-bb08-cf1ae8455a7e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:00.056190 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88bd42d8-4b91-4d58-b9ea-899dbb4cac35) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:00.090252 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88bd42d8-4b91-4d58-b9ea-899dbb4cac35) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '3625ab3c-1eed-48fc-94c0-694f99a884df', 'volume_name': 'volume-3625ab3c-1eed-48fc-94c0-694f99a884df', 'volume_size': 2, 'source_volid': 'bb5dc462-0184-4114-be06-448971acfe05', 'source_volstatus': 'available'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:00.095709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (609f532a-2ab3-4c6d-8e3d-f23831df31a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:00.100455 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (609f532a-2ab3-4c6d-8e3d-f23831df31a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:00.108162 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (864975ae-9d8f-4f3f-ad9c-34013a398aec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:00.109754 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Volume 3625ab3c-1eed-48fc-94c0-694f99a884df: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-3625ab3c-1eed-48fc-94c0-694f99a884df', 'volume_size': 2, 'source_volid': 'bb5dc462-0184-4114-be06-448971acfe05', 'source_volstatus': 'available'} May 14 21:28:00.138638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:00.521793 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] creating snapshot='volume-3625ab3c-1eed-48fc-94c0-694f99a884df.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:28:03.562470 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] cloning 'volume-bb5dc462-0184-4114-be06-448971acfe05@volume-3625ab3c-1eed-48fc-94c0-694f99a884df.clone_snap' to 'volume-3625ab3c-1eed-48fc-94c0-694f99a884df' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:28:03.567557 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:28:03.568435 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:03.606995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:03.640833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:28:05.375437 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Volume volume-bb5dc462-0184-4114-be06-448971acfe05 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:28:05.376875 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:28:05.378064 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:28:05.973806 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] resize volume 'volume-3625ab3c-1eed-48fc-94c0-694f99a884df' from 1 to 2 {{(pid=123870) _extend_if_required /opt/stack/cinder/cinder/volume/drivers/rbd.py:745}} May 14 21:28:05.974520 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:07.781284 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:28:07.785746 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (864975ae-9d8f-4f3f-ad9c-34013a398aec) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3625ab3c-1eed-48fc-94c0-694f99a884df', 'volume_size': 2, 'source_volid': 'bb5dc462-0184-4114-be06-448971acfe05', 'source_volstatus': 'available'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:07.790857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b6d34da9-8d22-427a-a5a1-97672eb1466e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:07.882283 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Volume volume-3625ab3c-1eed-48fc-94c0-694f99a884df (3625ab3c-1eed-48fc-94c0-694f99a884df): created successfully May 14 21:28:07.886014 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b6d34da9-8d22-427a-a5a1-97672eb1466e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:07.892150 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Flow 'volume_create_manager' (789fd4a3-11bd-471b-b376-77fe9b817f4c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:28:08.001908 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-cf300eff-a297-43cd-94ad-15099ba3a755 tempest-VolumesCloneTest-218403688 None] Created volume successfully. May 14 21:28:08.430483 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-3625ab3c-1eed-48fc-94c0-694f99a884df-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:28:08.535843 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:08.769926 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:28:09.020653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] deleting rbd volume volume-3625ab3c-1eed-48fc-94c0-694f99a884df {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:28:15.129057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:28:15.460362 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] Volume volume-bb5dc462-0184-4114-be06-448971acfe05 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:28:15.460780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] deleting parent snapshot volume-3625ab3c-1eed-48fc-94c0-694f99a884df.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:28:18.615657 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] Created reservations ['1cad7b38-f2af-4468-b1f6-c96fc6933b56', 'b9ffba5d-bd2b-4a56-afb9-9aa9bf96cef5', '287d9dd7-e1f8-44c3-9c33-6d14b18ec3b3', '2ce32f64-d376-49c5-80ea-cc8f2cba56ab'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:28:18.882621 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:18.919238 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:18.981531 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:28:18.989507 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] Deleted volume successfully. May 14 21:28:19.049743 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-920df6cb-48c6-41ad-8ddd-2861203b1e89 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-3625ab3c-1eed-48fc-94c0-694f99a884df-delete_volume'" released by "delete_volume" :: held 10.619s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:28:19.423133 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-bb5dc462-0184-4114-be06-448971acfe05-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:28:19.593922 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:19.829535 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:28:19.830949 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] Volume volume-bb5dc462-0184-4114-be06-448971acfe05 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:28:20.054087 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] deleting rbd volume volume-bb5dc462-0184-4114-be06-448971acfe05 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:28:25.990030 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] Created reservations ['1de2786c-3782-45e0-8e22-4d3859c5a00f', 'e5bb4b06-15bd-4df6-a715-cfc0a4a1678d', 'ccc78529-360e-486d-95ef-ff10000b1150', '9ecfae5d-d629-4a4a-a64a-b5c06eea324b'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:28:26.138887 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:26.273809 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:26.322676 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:28:26.336013 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] Deleted volume successfully. May 14 21:28:26.424858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cf4f06d2-9d64-4405-997a-10fa3fd92462 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-bb5dc462-0184-4114-be06-448971acfe05-delete_volume'" released by "delete_volume" :: held 6.995s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:28:27.020639 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-9d841ce2-7872-4287-9cab-5db012a7caee-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:28:27.152177 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:27.379969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:28:27.580823 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] deleting rbd volume volume-9d841ce2-7872-4287-9cab-5db012a7caee {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:28:27.700262 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:28:27.723823 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Flow 'volume_create_manager' (a9e6b8ce-988d-4343-bc14-da5293f101c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:28:27.731166 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (aa0972cd-4e15-41d2-801d-72564043fe19) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:27.751677 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (aa0972cd-4e15-41d2-801d-72564043fe19) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:28:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-1596564146',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d520492e-c3a1-42a9-a065-6b741defc4c0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4989c480608c4b86bd1cd7615fba4d20',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:28:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:28:27Z,use_quota=True,user_id='04e48304473d40ebb8c0c5386c4c3592',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:27.765475 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8bec1002-c222-4bb6-ba12-66f1d9e269ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:27.770477 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8bec1002-c222-4bb6-ba12-66f1d9e269ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:27.776466 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a40b02c3-4934-429d-a80c-5bb241262efe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:27.781574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a40b02c3-4934-429d-a80c-5bb241262efe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd520492e-c3a1-42a9-a065-6b741defc4c0', 'volume_name': 'volume-d520492e-c3a1-42a9-a065-6b741defc4c0', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:27.797015 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (55edd742-11b9-4c11-8410-bcca33636f01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:27.802381 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (55edd742-11b9-4c11-8410-bcca33636f01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:27.813002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ed6d8d79-3493-4eb3-b186-4aec799182c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:27.816363 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Volume d520492e-c3a1-42a9-a065-6b741defc4c0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d520492e-c3a1-42a9-a065-6b741defc4c0', 'volume_size': 1} May 14 21:28:27.816964 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] creating volume 'volume-d520492e-c3a1-42a9-a065-6b741defc4c0' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:28:27.819044 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:28.748344 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:28:28.749456 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:28:28.754407 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ed6d8d79-3493-4eb3-b186-4aec799182c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d520492e-c3a1-42a9-a065-6b741defc4c0', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:28.759217 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2930747-1552-4abf-8270-2ddaf36f72bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:28.887052 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Volume volume-d520492e-c3a1-42a9-a065-6b741defc4c0 (d520492e-c3a1-42a9-a065-6b741defc4c0): created successfully May 14 21:28:28.891176 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2930747-1552-4abf-8270-2ddaf36f72bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:28.897000 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Flow 'volume_create_manager' (a9e6b8ce-988d-4343-bc14-da5293f101c6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:28:29.021417 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ceda355a-628c-4186-8f5a-891a619ec41d tempest-AttachVolumeShelveTestJSON-565426114 None] Created volume successfully. May 14 21:28:33.051090 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-9bc20a05-9740-4fa4-b99c-e35f0e390973 req-d508a7e2-6918-400c-9f28-77b5f2a86f83 tempest-AttachVolumeNegativeTest-1286916119 None] Terminate volume connection completed successfully. May 14 21:28:33.055759 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-9bc20a05-9740-4fa4-b99c-e35f0e390973 req-d508a7e2-6918-400c-9f28-77b5f2a86f83 tempest-AttachVolumeNegativeTest-1286916119 None] Deleting attachment a362418e-f1c7-4b9c-ba9e-e361e3e6cddf. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:28:33.058467 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:28:33.328274 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] deleting parent snapshot volume-9d841ce2-7872-4287-9cab-5db012a7caee.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:28:36.813815 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] Created reservations ['2f791378-15f9-4c35-95eb-903f7b769821', 'a9cfc709-bfc8-4975-aee0-4c5b518fb192', '2066208e-8573-4519-a8a7-0a7a487df1c8', '52b38a69-6ac9-4e10-b7c6-45151b75dd21'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:28:37.013615 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:37.085168 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:37.127177 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:28:37.132100 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] Deleted volume successfully. May 14 21:28:37.239499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-95d5b818-1fca-419d-935a-37d1af80f4cf tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-9d841ce2-7872-4287-9cab-5db012a7caee-delete_volume'" released by "delete_volume" :: held 10.217s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:28:38.068627 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-ec66fdb7-4204-4ce7-befa-e3acd2c72003-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:28:38.155336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:38.453445 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:28:38.724005 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] deleting rbd volume volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:28:46.145296 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:28:46.368283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] deleting parent snapshot volume-ec66fdb7-4204-4ce7-befa-e3acd2c72003.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:28:51.098067 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] Created reservations ['691795ec-dfe2-4635-ac5b-d48a40375074', 'fd1bf995-f6ea-4df0-a70e-d63143cf540c', '7e2b7485-6244-4212-8e25-c6c7473d74f7', '1afd750d-f8a9-4b9d-8c3a-c2d91bb6fe0a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:28:51.322064 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:51.464646 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:51.523080 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:28:51.527311 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] Deleted volume successfully. May 14 21:28:51.650502 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-eac2f509-3ed1-48de-b237-8441743c1c52 tempest-VolumesCloneTest-218403688 None] Lock "b'cinder-ec66fdb7-4204-4ce7-befa-e3acd2c72003-delete_volume'" released by "delete_volume" :: held 13.617s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:28:56.734447 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:28:56.775267 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Flow 'volume_create_manager' (a55ca87c-25fc-4be6-be83-3702c86ccbda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:28:56.793268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b6edd769-c44b-4e2e-82c5-d867b4ad6fd3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:56.815663 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b6edd769-c44b-4e2e-82c5-d867b4ad6fd3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:28:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-1522876647',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=20537a77-cb10-4dbd-9525-738968d3df7e,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='939c7de6596640f1b2b8081775020666',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:28:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:28:57Z,use_quota=True,user_id='cf3c57181efa47658e72ebc678ad3035',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:56.824770 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f013a59b-4666-46d8-9954-06ea8a41f8ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:56.829510 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f013a59b-4666-46d8-9954-06ea8a41f8ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:56.835684 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fbdcfa6a-42c8-4a34-bd3f-953491dc5ee2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:56.841699 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fbdcfa6a-42c8-4a34-bd3f-953491dc5ee2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '20537a77-cb10-4dbd-9525-738968d3df7e', 'volume_name': 'volume-20537a77-cb10-4dbd-9525-738968d3df7e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:56.850366 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (abb9b6c1-0df1-4e67-b1b4-146657206452) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:56.854555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (abb9b6c1-0df1-4e67-b1b4-146657206452) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:56.860535 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca6cfa59-cf79-450a-9b52-eb45ebfcf289) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:56.862530 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Volume 20537a77-cb10-4dbd-9525-738968d3df7e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-20537a77-cb10-4dbd-9525-738968d3df7e', 'volume_size': 1} May 14 21:28:56.862936 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] creating volume 'volume-20537a77-cb10-4dbd-9525-738968d3df7e' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:28:56.863768 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:28:58.480375 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:28:58.481462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:28:58.485715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca6cfa59-cf79-450a-9b52-eb45ebfcf289) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-20537a77-cb10-4dbd-9525-738968d3df7e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:58.494959 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7ff86002-0e51-4149-9d2e-69cae4f3f13d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:28:58.584332 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Volume volume-20537a77-cb10-4dbd-9525-738968d3df7e (20537a77-cb10-4dbd-9525-738968d3df7e): created successfully May 14 21:28:58.588173 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7ff86002-0e51-4149-9d2e-69cae4f3f13d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:28:58.594222 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Flow 'volume_create_manager' (a55ca87c-25fc-4be6-be83-3702c86ccbda) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:28:58.683094 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fcea3ed4-a42c-4099-9469-d8088aa9df49 tempest-VolumesListTestJSON-918772999 None] Created volume successfully. May 14 21:29:00.926534 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:29:00.987063 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Flow 'volume_create_manager' (312f6279-99ec-4cf4-ad55-36844a9f8e11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:00.994835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (928964bb-d815-4e1e-89ea-3fbfd04e06fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:01.012488 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (928964bb-d815-4e1e-89ea-3fbfd04e06fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:29:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-1500798162',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=47f3805a-cb78-46b5-ba6d-f6d5a067cb4c,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='939c7de6596640f1b2b8081775020666',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:29:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:29:01Z,use_quota=True,user_id='cf3c57181efa47658e72ebc678ad3035',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:01.019652 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f67fade5-3263-4127-a1aa-9769937903d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:01.023884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f67fade5-3263-4127-a1aa-9769937903d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:01.029169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (26f00728-41d4-4594-bd79-4f99bf2b109e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:01.034543 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (26f00728-41d4-4594-bd79-4f99bf2b109e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '47f3805a-cb78-46b5-ba6d-f6d5a067cb4c', 'volume_name': 'volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:01.040197 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (756b66ce-a102-45ce-a641-428b74e8d219) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:01.044416 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (756b66ce-a102-45ce-a641-428b74e8d219) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:01.056697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (47717ce1-19fc-4c5b-9fef-246d39b69b96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:01.057831 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Volume 47f3805a-cb78-46b5-ba6d-f6d5a067cb4c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c', 'volume_size': 1} May 14 21:29:01.062086 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] creating volume 'volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:29:01.062086 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:03.025728 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:29:03.027787 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:29:03.035918 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (47717ce1-19fc-4c5b-9fef-246d39b69b96) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:03.040673 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8f15e5b7-11a1-41bf-a094-84bdca0f2bc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:03.218089 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Volume volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c (47f3805a-cb78-46b5-ba6d-f6d5a067cb4c): created successfully May 14 21:29:03.222895 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8f15e5b7-11a1-41bf-a094-84bdca0f2bc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:03.232708 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Flow 'volume_create_manager' (312f6279-99ec-4cf4-ad55-36844a9f8e11) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:03.321471 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-eae96243-4a7c-48ae-8671-2a6e134de45c tempest-VolumesListTestJSON-918772999 None] Created volume successfully. May 14 21:29:03.516625 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:29:03.538298 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Flow 'volume_create_manager' (aa46d289-7beb-4acc-be3e-2433d009ad7b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:03.544923 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (852703fb-f3f8-4bc2-9e9d-72093dbab1ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:03.561852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (852703fb-f3f8-4bc2-9e9d-72093dbab1ab) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:29:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-471006163',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fe4b9b45-3bd4-47a6-bd85-901d2fa90799,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6641b342c1d440aea451919f391bc5db',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:29:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:29:03Z,use_quota=True,user_id='5586a3a946a0490498c42cb71f39ce1e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:03.567503 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b8425e77-b4e0-456f-9d28-6478eda845a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:03.571877 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b8425e77-b4e0-456f-9d28-6478eda845a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:03.577310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (30460e58-2cd0-4536-a0b1-293f88828f56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:03.582255 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (30460e58-2cd0-4536-a0b1-293f88828f56) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'volume_name': 'volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:03.595606 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80d287c1-8f6c-4da0-8a7f-7756e6a93943) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:03.601382 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80d287c1-8f6c-4da0-8a7f-7756e6a93943) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:03.606808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f4c9426-b306-4cff-b943-5f698c86f6cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:03.608216 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Volume fe4b9b45-3bd4-47a6-bd85-901d2fa90799: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'volume_size': 1} May 14 21:29:03.608808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] creating volume 'volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:29:03.609653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:03.664276 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:29:03.668514 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:03.769035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:03.826364 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:29:04.683262 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:29:04.684344 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:29:04.691330 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f4c9426-b306-4cff-b943-5f698c86f6cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:04.711053 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (51300b10-1883-4039-ad15-9357a69d1299) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:04.824878 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Volume volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799 (fe4b9b45-3bd4-47a6-bd85-901d2fa90799): created successfully May 14 21:29:04.834195 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (51300b10-1883-4039-ad15-9357a69d1299) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:04.852814 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Flow 'volume_create_manager' (aa46d289-7beb-4acc-be3e-2433d009ad7b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:04.956275 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-510fd53a-d940-4065-ad5f-6bd46a9c8b0e tempest-AttachVolumeTestJSON-1845080071 None] Created volume successfully. May 14 21:29:04.978855 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:29:05.007888 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Flow 'volume_create_manager' (79f792ba-12f2-4b1a-822e-57e479737099) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:05.014921 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e78a58e4-ab6b-4330-b21a-60f4a24ee9f6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:05.031061 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e78a58e4-ab6b-4330-b21a-60f4a24ee9f6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:29:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-641803644',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=306f23ac-85d1-42e2-be92-ad49fe0d3501,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='939c7de6596640f1b2b8081775020666',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:29:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:29:05Z,use_quota=True,user_id='cf3c57181efa47658e72ebc678ad3035',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:05.037396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a08a0530-2e96-4fdf-98e1-95e216c0453f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:05.041550 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a08a0530-2e96-4fdf-98e1-95e216c0453f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:05.046648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (856b2098-d3bc-4c5b-bf6a-76eea5d47ebd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:05.051355 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (856b2098-d3bc-4c5b-bf6a-76eea5d47ebd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '306f23ac-85d1-42e2-be92-ad49fe0d3501', 'volume_name': 'volume-306f23ac-85d1-42e2-be92-ad49fe0d3501', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:05.056918 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (924293e8-2953-44a2-bde6-2a6684c2c9cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:05.060848 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (924293e8-2953-44a2-bde6-2a6684c2c9cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:05.065811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f524475d-8d7a-43f7-ac5a-3e2facef9182) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:05.067849 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Volume 306f23ac-85d1-42e2-be92-ad49fe0d3501: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-306f23ac-85d1-42e2-be92-ad49fe0d3501', 'volume_size': 1} May 14 21:29:05.067849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] creating volume 'volume-306f23ac-85d1-42e2-be92-ad49fe0d3501' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:29:05.068092 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:06.287894 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f0b1e0b4-c472-4192-9f11-22f5d472e774 req-e646a9ef-def8-4c07-af6a-943d5f7817b5 tempest-AttachVolumeTestJSON-1845080071 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:29:06.361972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:29:06.362806 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:29:06.367872 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f524475d-8d7a-43f7-ac5a-3e2facef9182) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-306f23ac-85d1-42e2-be92-ad49fe0d3501', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:06.372841 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e6b5f242-6e0a-4972-b2cf-e8fcf26a6992) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:06.469395 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Volume volume-306f23ac-85d1-42e2-be92-ad49fe0d3501 (306f23ac-85d1-42e2-be92-ad49fe0d3501): created successfully May 14 21:29:06.473764 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e6b5f242-6e0a-4972-b2cf-e8fcf26a6992) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:06.479513 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Flow 'volume_create_manager' (79f792ba-12f2-4b1a-822e-57e479737099) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:06.607436 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e6aa4fe5-884f-4ad8-a9c8-afb8ebb4e3f1 tempest-VolumesListTestJSON-918772999 None] Created volume successfully. May 14 21:29:06.957022 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f0b1e0b4-c472-4192-9f11-22f5d472e774 req-e646a9ef-def8-4c07-af6a-943d5f7817b5 tempest-AttachVolumeTestJSON-1845080071 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.668s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:29:06.957235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-f0b1e0b4-c472-4192-9f11-22f5d472e774 req-e646a9ef-def8-4c07-af6a-943d5f7817b5 tempest-AttachVolumeTestJSON-1845080071 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:29:07.086801 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f0b1e0b4-c472-4192-9f11-22f5d472e774 req-e646a9ef-def8-4c07-af6a-943d5f7817b5 tempest-AttachVolumeTestJSON-1845080071 None] Connection info returned from driver {'name': 'volumes/volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'fe4b9b45-3bd4-47a6-bd85-901d2fa90799', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '05f73719-2c8f-4e5f-8862-8e5cf21997f3'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:29:07.215291 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f0b1e0b4-c472-4192-9f11-22f5d472e774 req-e646a9ef-def8-4c07-af6a-943d5f7817b5 tempest-AttachVolumeTestJSON-1845080071 None] attachment_update completed successfully. May 14 21:29:09.992708 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] Lock "b'cinder-306f23ac-85d1-42e2-be92-ad49fe0d3501-delete_volume'" acquired by "delete_volume" :: waited 0.018s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:29:10.094010 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:10.521271 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:29:10.521271 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] Volume volume-306f23ac-85d1-42e2-be92-ad49fe0d3501 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:29:10.777365 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] deleting rbd volume volume-306f23ac-85d1-42e2-be92-ad49fe0d3501 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:29:11.863993 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-b5c326b8-6d43-4a76-bc80-8413d46498c4 req-3b70922e-aec9-404d-8602-5898da65c77f tempest-AttachVolumeShelveTestJSON-565426114 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:29:12.524525 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-b5c326b8-6d43-4a76-bc80-8413d46498c4 req-3b70922e-aec9-404d-8602-5898da65c77f tempest-AttachVolumeShelveTestJSON-565426114 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.660s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:29:12.525657 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-b5c326b8-6d43-4a76-bc80-8413d46498c4 req-3b70922e-aec9-404d-8602-5898da65c77f tempest-AttachVolumeShelveTestJSON-565426114 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d520492e-c3a1-42a9-a065-6b741defc4c0', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'd520492e-c3a1-42a9-a065-6b741defc4c0', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:29:12.600754 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-b5c326b8-6d43-4a76-bc80-8413d46498c4 req-3b70922e-aec9-404d-8602-5898da65c77f tempest-AttachVolumeShelveTestJSON-565426114 None] Connection info returned from driver {'name': 'volumes/volume-d520492e-c3a1-42a9-a065-6b741defc4c0', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd520492e-c3a1-42a9-a065-6b741defc4c0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '2ef77bc2-da24-471e-bd18-1b3e4616133c'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:29:12.759495 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-b5c326b8-6d43-4a76-bc80-8413d46498c4 req-3b70922e-aec9-404d-8602-5898da65c77f tempest-AttachVolumeShelveTestJSON-565426114 None] attachment_update completed successfully. May 14 21:29:19.995616 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] Created reservations ['67ee6ed1-4820-4bce-8fc5-61216c20e9a7', 'b52ab25c-2a10-44dc-98c2-138aa1d4a1fc', 'd9da329b-0071-42c0-9789-9e8b62ae5289', '753cc4ba-fdbb-4bba-9d82-61b8773999a4'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:29:20.272154 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:20.296410 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:20.325099 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:29:20.329839 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] Deleted volume successfully. May 14 21:29:20.630500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e8247418-b046-4e17-8025-04a990b75743 tempest-VolumesListTestJSON-918772999 None] Lock "b'cinder-306f23ac-85d1-42e2-be92-ad49fe0d3501-delete_volume'" released by "delete_volume" :: held 10.634s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:29:20.894804 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] Lock "b'cinder-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:29:21.126814 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:21.828278 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:29:21.828809 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] Volume volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:29:22.475314 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] deleting rbd volume volume-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:29:30.509831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] Created reservations ['48d14379-7611-4f32-a26e-7d91a32017b7', '1d084d1d-2004-4364-bf34-4de0c8583467', 'a47f1734-6e5c-40d3-9a30-0dbcba8cd876', 'a95cc228-bdff-4b53-877d-f125627cdcb2'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:29:30.664185 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:30.686567 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:30.710339 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:29:30.714313 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] Deleted volume successfully. May 14 21:29:30.847006 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a9928b32-8126-40ee-9bf9-f3412f78426e tempest-VolumesListTestJSON-918772999 None] Lock "b'cinder-47f3805a-cb78-46b5-ba6d-f6d5a067cb4c-delete_volume'" released by "delete_volume" :: held 9.952s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:29:31.738834 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] Lock "b'cinder-20537a77-cb10-4dbd-9525-738968d3df7e-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:29:31.903793 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:32.211564 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:29:32.211996 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] Volume volume-20537a77-cb10-4dbd-9525-738968d3df7e is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:29:32.411962 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] deleting rbd volume volume-20537a77-cb10-4dbd-9525-738968d3df7e {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:29:40.411308 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] Created reservations ['63c07f57-f641-46c9-9b44-c4a47e0fee60', 'a7efd063-8fe2-4e48-a146-9d18ea423069', 'a0e1b41e-e70e-497e-a59b-15d841e03551', '6084fe50-54f1-49fb-ba80-86ed31c0705d'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:29:40.612457 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:40.634799 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:40.662642 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:29:40.673795 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] Deleted volume successfully. May 14 21:29:40.718284 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4846c278-0568-4062-bc15-16c78c020ae5 tempest-VolumesListTestJSON-918772999 None] Lock "b'cinder-20537a77-cb10-4dbd-9525-738968d3df7e-delete_volume'" released by "delete_volume" :: held 8.979s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:29:40.738764 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:29:40.758432 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Flow 'volume_create_manager' (a8897a06-d21d-4efa-b85a-da39162e7ac9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:40.765204 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f9863049-259d-4410-be61-fc761955a630) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:40.781097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f9863049-259d-4410-be61-fc761955a630) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:29:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1818711430',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e3d7bef2-cd53-4508-9fd9-b7613815d546,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8bd6f25048cd4798985e725f734f1845',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:29:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:29:41Z,use_quota=True,user_id='1e3390c9579a4f598c46f9417b4dd046',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:40.787127 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (30c28d84-d454-4644-9c28-65f6607236c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:40.791213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (30c28d84-d454-4644-9c28-65f6607236c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:40.796308 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (00028508-b3c8-4079-8ac6-ba1f763a669c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:40.800493 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (00028508-b3c8-4079-8ac6-ba1f763a669c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e3d7bef2-cd53-4508-9fd9-b7613815d546', 'volume_name': 'volume-e3d7bef2-cd53-4508-9fd9-b7613815d546', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:40.805393 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ee9b4cc2-d996-4abc-87cb-5f53c0f38f29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:40.809247 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ee9b4cc2-d996-4abc-87cb-5f53c0f38f29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:40.814144 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d68b259-9ac0-4a7a-9393-8dd1931db298) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:40.815211 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Volume e3d7bef2-cd53-4508-9fd9-b7613815d546: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e3d7bef2-cd53-4508-9fd9-b7613815d546', 'volume_size': 1} May 14 21:29:40.815638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] creating volume 'volume-e3d7bef2-cd53-4508-9fd9-b7613815d546' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:29:40.816427 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:42.042565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:29:42.045990 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:29:42.047768 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d68b259-9ac0-4a7a-9393-8dd1931db298) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e3d7bef2-cd53-4508-9fd9-b7613815d546', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:42.052742 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07d195ed-904b-4cd5-abb8-fb42f9d68a78) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:42.126739 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Volume volume-e3d7bef2-cd53-4508-9fd9-b7613815d546 (e3d7bef2-cd53-4508-9fd9-b7613815d546): created successfully May 14 21:29:42.130938 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07d195ed-904b-4cd5-abb8-fb42f9d68a78) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:42.138722 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Flow 'volume_create_manager' (a8897a06-d21d-4efa-b85a-da39162e7ac9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:42.222436 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-942b230a-0b87-48b8-b35b-528bab2354ac tempest-AttachVolumeNegativeTest-1286916119 None] Created volume successfully. May 14 21:29:44.246587 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f20b2976-ec20-4397-bd0c-0360aa137492 req-e43c051b-c33d-4270-9179-f6cc9b8865d6 tempest-AttachVolumeNegativeTest-1286916119 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:29:44.977419 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-f20b2976-ec20-4397-bd0c-0360aa137492 req-e43c051b-c33d-4270-9179-f6cc9b8865d6 tempest-AttachVolumeNegativeTest-1286916119 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.731s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:29:44.978487 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-f20b2976-ec20-4397-bd0c-0360aa137492 req-e43c051b-c33d-4270-9179-f6cc9b8865d6 tempest-AttachVolumeNegativeTest-1286916119 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e3d7bef2-cd53-4508-9fd9-b7613815d546', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e3d7bef2-cd53-4508-9fd9-b7613815d546', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:29:45.074628 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-f20b2976-ec20-4397-bd0c-0360aa137492 req-e43c051b-c33d-4270-9179-f6cc9b8865d6 tempest-AttachVolumeNegativeTest-1286916119 None] Connection info returned from driver {'name': 'volumes/volume-e3d7bef2-cd53-4508-9fd9-b7613815d546', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e3d7bef2-cd53-4508-9fd9-b7613815d546', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '81c0573a-bd0d-43b4-ab96-5deb85bcaf65'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:29:45.205893 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-f20b2976-ec20-4397-bd0c-0360aa137492 req-e43c051b-c33d-4270-9179-f6cc9b8865d6 tempest-AttachVolumeNegativeTest-1286916119 None] attachment_update completed successfully. May 14 21:29:46.664868 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:29:46.685060 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (d3fe5ebc-1e79-4dce-a6cb-fcc5558606ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:46.693260 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (42a333d4-62b5-44f5-9aef-669ee6c03ec1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:46.711417 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (42a333d4-62b5-44f5-9aef-669ee6c03ec1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:29:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-869721996',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c221be08-99e4-4f09-a858-6fffe81fd931,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='162610ac2697486d8a05f41713cc5ba2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:29:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:29:46Z,use_quota=True,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:46.717462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (54569cd1-4751-4d85-84ed-539f24212e6e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:46.722516 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (54569cd1-4751-4d85-84ed-539f24212e6e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:46.727821 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51d6b9bd-a8d1-4b77-b3fe-ad67fc77b817) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:46.733058 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51d6b9bd-a8d1-4b77-b3fe-ad67fc77b817) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c221be08-99e4-4f09-a858-6fffe81fd931', 'volume_name': 'volume-c221be08-99e4-4f09-a858-6fffe81fd931', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:46.738632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b27ab77d-0aef-4e8b-a40c-521d3fe86452) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:46.742653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b27ab77d-0aef-4e8b-a40c-521d3fe86452) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:46.747373 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6ed87a25-f2a7-4216-88a5-fd82e2513136) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:46.748503 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume c221be08-99e4-4f09-a858-6fffe81fd931: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c221be08-99e4-4f09-a858-6fffe81fd931', 'volume_size': 1} May 14 21:29:46.748919 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] creating volume 'volume-c221be08-99e4-4f09-a858-6fffe81fd931' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:29:46.749752 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:47.832498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:29:47.833151 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:29:47.836526 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6ed87a25-f2a7-4216-88a5-fd82e2513136) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c221be08-99e4-4f09-a858-6fffe81fd931', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:47.841641 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c24f8fcb-ab60-4719-b1ac-83ed979bc05b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:29:47.908170 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-c221be08-99e4-4f09-a858-6fffe81fd931 (c221be08-99e4-4f09-a858-6fffe81fd931): created successfully May 14 21:29:47.912048 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c24f8fcb-ab60-4719-b1ac-83ed979bc05b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:29:47.917648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (d3fe5ebc-1e79-4dce-a6cb-fcc5558606ea) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:29:48.045548 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6c385bc0-16c3-4e45-ab39-bc485c53e785 tempest-VolumesSnapshotTestJSON-1533504236 None] Created volume successfully. May 14 21:29:49.223865 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-04407191-9ffe-453a-9cd6-7db2bcf09142 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:49.686100 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-145ab0a0-454f-4745-b798-b71ab9a9e951 req-4ebbf454-e962-48a5-8fe0-bab645292848 tempest-AttachVolumeNegativeTest-1286916119 None] Terminate volume connection completed successfully. May 14 21:29:49.686100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-145ab0a0-454f-4745-b798-b71ab9a9e951 req-4ebbf454-e962-48a5-8fe0-bab645292848 tempest-AttachVolumeNegativeTest-1286916119 None] Deleting attachment 81c0573a-bd0d-43b4-ab96-5deb85bcaf65. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:29:52.480401 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-04407191-9ffe-453a-9cd6-7db2bcf09142 tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:29:54.139046 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.driver [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Creating a new backup for volume volume-c221be08-99e4-4f09-a858-6fffe81fd931. {{(pid=123870) _get_backup_volume_temp_volume /opt/stack/cinder/cinder/volume/driver.py:1213}} May 14 21:29:54.245281 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] cloning volumes/volume-c221be08-99e4-4f09-a858-6fffe81fd931@snapshot-f9966700-2a70-49c4-a5bc-d0c19d8baa1a to volume-d4a3485e-9088-48b6-ac8b-2122497b6eae {{(pid=123870) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}} May 14 21:29:54.245281 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:54.334442 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:56.290356 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:29:56.294850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:29:56.296040 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:29:56.343674 ncdv-master-1 cinder-volume[123870]: WARNING cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues May 14 21:29:56.524989 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:29:53Z,data_timestamp=2022-05-14T21:29:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-188718327',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=fe83b34c-76e8-421b-a507-f43e0f25e8ce,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='162610ac2697486d8a05f41713cc5ba2',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='f9966700-2a70-49c4-a5bc-d0c19d8baa1a',status='creating',temp_snapshot_id=None,temp_volume_id='d4a3485e-9088-48b6-ac8b-2122497b6eae',updated_at=None,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_id=c221be08-99e4-4f09-a858-6fffe81fd931) May 14 21:29:56.528766 ncdv-master-1 cinder-volume[123870]: INFO cinder.rpc [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Automatically selected cinder-backup objects version 1.39 as minimum service version. May 14 21:29:56.533797 ncdv-master-1 cinder-volume[123870]: INFO cinder.rpc [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Automatically selected cinder-backup RPC version 2.3 as minimum service version. May 14 21:29:56.534720 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] continue_backup in rpcapi backup_id fe83b34c-76e8-421b-a507-f43e0f25e8ce {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:29:57.811169 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-b2069536-118b-43fb-9aa7-096505b19edf req-6b6bbc60-192b-4267-89f5-13c1a6b34548 tempest-AttachVolumeTestJSON-1845080071 None] Terminate volume connection completed successfully. May 14 21:29:57.814125 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-b2069536-118b-43fb-9aa7-096505b19edf req-6b6bbc60-192b-4267-89f5-13c1a6b34548 tempest-AttachVolumeTestJSON-1845080071 None] Deleting attachment 05f73719-2c8f-4e5f-8862-8e5cf21997f3. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:30:02.040249 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:30:02.824363 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.784s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:30:02.828976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d4a3485e-9088-48b6-ac8b-2122497b6eae', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'd4a3485e-9088-48b6-ac8b-2122497b6eae', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:30:02.840196 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Initialize volume connection completed successfully. May 14 21:30:03.666709 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:30:03.669971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:03.743123 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:03.777776 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:30:13.978261 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d605fde2-b157-440c-b17b-8fb14edd1a97 req-7d906416-71ec-491c-be14-9d202fd427c1 tempest-AttachVolumeShelveTestJSON-565426114 None] Terminate volume connection completed successfully. May 14 21:30:13.978978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d605fde2-b157-440c-b17b-8fb14edd1a97 req-7d906416-71ec-491c-be14-9d202fd427c1 tempest-AttachVolumeShelveTestJSON-565426114 None] Deleting attachment 2ef77bc2-da24-471e-bd18-1b3e4616133c. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:30:31.301973 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-84c0e141-78b7-49b9-a9e9-1dfcbbbd41ce req-b402f2d8-315f-4420-ad19-b90e42a88a07 tempest-TaggedBootDevicesTest_v242-387890351 None] Terminate volume connection completed successfully. May 14 21:30:31.301973 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-84c0e141-78b7-49b9-a9e9-1dfcbbbd41ce req-b402f2d8-315f-4420-ad19-b90e42a88a07 tempest-TaggedBootDevicesTest_v242-387890351 None] Deleting attachment 808673fc-7787-4534-b341-aa06336bb3ae. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:30:31.560386 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-84c0e141-78b7-49b9-a9e9-1dfcbbbd41ce req-5784bb26-281d-4c29-a80b-3f5ead55897d tempest-TaggedBootDevicesTest_v242-387890351 None] Terminate volume connection completed successfully. May 14 21:30:31.560386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-84c0e141-78b7-49b9-a9e9-1dfcbbbd41ce req-5784bb26-281d-4c29-a80b-3f5ead55897d tempest-TaggedBootDevicesTest_v242-387890351 None] Deleting attachment d4a41f9c-417b-4845-bc6e-9b99f08871bf. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:30:31.829699 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-84c0e141-78b7-49b9-a9e9-1dfcbbbd41ce req-8cefd651-f83b-4348-b285-b8376e71d0af tempest-TaggedBootDevicesTest_v242-387890351 None] Terminate volume connection completed successfully. May 14 21:30:31.832927 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-84c0e141-78b7-49b9-a9e9-1dfcbbbd41ce req-8cefd651-f83b-4348-b285-b8376e71d0af tempest-TaggedBootDevicesTest_v242-387890351 None] Deleting attachment 46eb12dc-9f22-43a4-9b76-684627940fdf. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:30:41.178300 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db-delete_volume'" acquired by "delete_volume" :: waited 0.020s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:30:41.324007 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:42.645829 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:30:42.646478 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:30:42.983453 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] deleting rbd volume volume-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:30:51.515035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:30:51.534933 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Flow 'volume_create_manager' (7e9bd4a6-8ddb-41a9-848e-c8c3f2970ee6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:30:51.541769 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0999f736-61cf-45dc-8efd-fc453a487869) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:30:51.557527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0999f736-61cf-45dc-8efd-fc453a487869) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:30:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1430439225',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b941c16b-80e2-4f26-bb91-adb28379cf31,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8bd6f25048cd4798985e725f734f1845',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:30:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:30:51Z,use_quota=True,user_id='1e3390c9579a4f598c46f9417b4dd046',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:30:51.563565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e5bbf230-8703-475c-9189-d37e2ec30bf2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:30:51.567679 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e5bbf230-8703-475c-9189-d37e2ec30bf2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:30:51.572698 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51da2ddf-72c0-4319-b111-c1876d1cd038) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:30:51.577735 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51da2ddf-72c0-4319-b111-c1876d1cd038) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b941c16b-80e2-4f26-bb91-adb28379cf31', 'volume_name': 'volume-b941c16b-80e2-4f26-bb91-adb28379cf31', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:30:51.583240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f539223e-ff8b-44c8-baa2-5885cc302057) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:30:51.587233 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f539223e-ff8b-44c8-baa2-5885cc302057) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:30:51.591979 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (967d019f-707e-4ab3-8fc4-75f81561d9a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:30:51.593434 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Volume b941c16b-80e2-4f26-bb91-adb28379cf31: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b941c16b-80e2-4f26-bb91-adb28379cf31', 'volume_size': 1} May 14 21:30:51.593834 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] creating volume 'volume-b941c16b-80e2-4f26-bb91-adb28379cf31' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:30:51.594638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:53.031114 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:30:53.032446 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:30:53.036022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (967d019f-707e-4ab3-8fc4-75f81561d9a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b941c16b-80e2-4f26-bb91-adb28379cf31', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:30:53.040699 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (38657aa7-9a43-461e-b9f1-988081598b37) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:30:53.106993 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Volume volume-b941c16b-80e2-4f26-bb91-adb28379cf31 (b941c16b-80e2-4f26-bb91-adb28379cf31): created successfully May 14 21:30:53.111136 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (38657aa7-9a43-461e-b9f1-988081598b37) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:30:53.116986 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Flow 'volume_create_manager' (7e9bd4a6-8ddb-41a9-848e-c8c3f2970ee6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:30:53.234611 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c7626aa1-22c2-402a-bfe2-788b6c17f95b tempest-AttachVolumeNegativeTest-1286916119 None] Created volume successfully. May 14 21:30:54.155035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] Created reservations ['8dce22a8-3a08-462d-b2ad-224559be29bc', '6b366628-ecb9-4022-bf49-c511cb9e24ea', '12c07000-7b48-40a9-b005-1beeb1ebfbf1', '81ae1bc3-8a95-4557-8c03-4759aac85c76'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:30:54.397792 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:54.443714 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:54.475087 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:30:54.496608 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] Deleted volume successfully. May 14 21:30:54.504779 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-b79a726c-80d5-42bd-8f23-9d79e54ac0a1 req-d1858ff5-a6fe-4d46-b543-a3cab35f2c15 tempest-AttachVolumeNegativeTest-1286916119 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:30:54.600162 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-326a483d-7d7c-4b02-91f7-f8bb370eb346 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-e6ef1b51-cb2d-47e8-b6cb-1d7a4cd951db-delete_volume'" released by "delete_volume" :: held 13.421s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:30:55.351975 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-b79a726c-80d5-42bd-8f23-9d79e54ac0a1 req-d1858ff5-a6fe-4d46-b543-a3cab35f2c15 tempest-AttachVolumeNegativeTest-1286916119 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.847s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:30:55.352710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-b79a726c-80d5-42bd-8f23-9d79e54ac0a1 req-d1858ff5-a6fe-4d46-b543-a3cab35f2c15 tempest-AttachVolumeNegativeTest-1286916119 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b941c16b-80e2-4f26-bb91-adb28379cf31', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'b941c16b-80e2-4f26-bb91-adb28379cf31', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:30:55.402648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-5ab5558f-588e-4b04-84f6-6f28c0048e77-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:30:55.430273 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-b79a726c-80d5-42bd-8f23-9d79e54ac0a1 req-d1858ff5-a6fe-4d46-b543-a3cab35f2c15 tempest-AttachVolumeNegativeTest-1286916119 None] Connection info returned from driver {'name': 'volumes/volume-b941c16b-80e2-4f26-bb91-adb28379cf31', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b941c16b-80e2-4f26-bb91-adb28379cf31', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5e35f9c4-a036-4085-ae42-e44c3d12af61'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:30:55.503312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:30:55.832625 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:30:55.833152 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] Volume volume-5ab5558f-588e-4b04-84f6-6f28c0048e77 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:30:56.133614 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] deleting rbd volume volume-5ab5558f-588e-4b04-84f6-6f28c0048e77 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:30:56.171430 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-b79a726c-80d5-42bd-8f23-9d79e54ac0a1 req-d1858ff5-a6fe-4d46-b543-a3cab35f2c15 tempest-AttachVolumeNegativeTest-1286916119 None] attachment_update completed successfully. May 14 21:31:02.146187 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-3187cd81-fcbf-4a76-84f7-05dcf108d97f req-ab50d96a-4ba7-4ca6-8a10-992a29615742 tempest-AttachVolumeNegativeTest-1286916119 None] Terminate volume connection completed successfully. May 14 21:31:02.146942 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-3187cd81-fcbf-4a76-84f7-05dcf108d97f req-ab50d96a-4ba7-4ca6-8a10-992a29615742 tempest-AttachVolumeNegativeTest-1286916119 None] Deleting attachment 5e35f9c4-a036-4085-ae42-e44c3d12af61. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:31:03.665675 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:31:03.666330 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:03.689380 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:03.714925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:31:08.243710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] Created reservations ['155d5422-375b-4b34-9f4f-27a53c931f0d', 'c3cefac7-931d-4b37-b7c6-1549411f50df', '12c8f941-31cf-44ae-af77-d80417351e42', '6e765dac-b6f2-458a-9117-56fb98ad680e'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:31:08.440089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:08.475399 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:08.521403 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:31:08.529318 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] Deleted volume successfully. May 14 21:31:08.643790 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9f6fc392-a8f6-47dc-b308-b42916a69507 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-5ab5558f-588e-4b04-84f6-6f28c0048e77-delete_volume'" released by "delete_volume" :: held 13.241s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:31:09.562697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] Lock "b'cinder-b941c16b-80e2-4f26-bb91-adb28379cf31-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:31:09.623670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-40376eea-0c0f-488c-ae00-97b792b66c72-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:31:09.724332 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:10.089891 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:31:10.101166 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] Volume volume-b941c16b-80e2-4f26-bb91-adb28379cf31 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:31:10.195397 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] deleting rbd volume volume-b941c16b-80e2-4f26-bb91-adb28379cf31 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:31:10.240021 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:10.508450 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:31:10.749905 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] deleting rbd volume volume-40376eea-0c0f-488c-ae00-97b792b66c72 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:31:21.144326 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:31:21.164565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Flow 'volume_create_manager' (f8305494-c989-4e42-89c9-54105b1e902c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:31:21.171668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (48c199e1-76d9-4862-bb7e-96e83bb092e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:21.189225 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (48c199e1-76d9-4862-bb7e-96e83bb092e7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:31:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-999685785',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=11d02ad8-e260-47a5-8195-f9c8f8b8ea85,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4989c480608c4b86bd1cd7615fba4d20',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:31:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:31:21Z,use_quota=True,user_id='04e48304473d40ebb8c0c5386c4c3592',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:21.194783 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8530a422-25a6-4437-b2f2-f5836b505ac2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:21.200247 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8530a422-25a6-4437-b2f2-f5836b505ac2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:21.241233 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3441dc0-71f6-481a-956d-2c8e7de288fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:21.250399 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3441dc0-71f6-481a-956d-2c8e7de288fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '11d02ad8-e260-47a5-8195-f9c8f8b8ea85', 'volume_name': 'volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:21.268411 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b16c2305-d66b-43ba-85d4-e803d03d7a39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:21.275436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b16c2305-d66b-43ba-85d4-e803d03d7a39) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:21.281351 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (54e7239f-1210-4f04-9cde-c2ebb5ab5d58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:21.285086 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Volume 11d02ad8-e260-47a5-8195-f9c8f8b8ea85: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85', 'volume_size': 1} May 14 21:31:21.285399 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] creating volume 'volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:31:21.286279 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:22.815945 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:31:22.821731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:31:22.827624 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (54e7239f-1210-4f04-9cde-c2ebb5ab5d58) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:22.833032 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8aff6022-f40c-4c09-95c7-b96c7db7d0f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:22.913512 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Volume volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85 (11d02ad8-e260-47a5-8195-f9c8f8b8ea85): created successfully May 14 21:31:22.924025 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8aff6022-f40c-4c09-95c7-b96c7db7d0f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:22.927270 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Flow 'volume_create_manager' (f8305494-c989-4e42-89c9-54105b1e902c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:31:23.147462 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7e80fc55-bad3-4270-b1bf-6182e92d2a72 tempest-AttachVolumeShelveTestJSON-565426114 None] Created volume successfully. May 14 21:31:24.044915 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] Created reservations ['ed98b8b9-606b-43c2-92ea-70a696ace9cd', '37844d0a-f596-4b71-931b-7bb0959e99b8', 'b02ffe04-8158-4f6f-b54e-5a0f64a2c4d8', '95d2e995-2049-4abb-9061-dada4d358258'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:31:24.235314 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:24.287127 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:24.322261 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:31:24.334169 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] Deleted volume successfully. May 14 21:31:24.388340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e3f454f6-ff20-41a7-a170-50abb62269bb tempest-AttachVolumeNegativeTest-1286916119 None] Lock "b'cinder-b941c16b-80e2-4f26-bb91-adb28379cf31-delete_volume'" released by "delete_volume" :: held 14.825s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:31:24.745438 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] Lock "b'cinder-e3d7bef2-cd53-4508-9fd9-b7613815d546-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:31:24.898768 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:25.319819 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:31:25.321049 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] Volume volume-e3d7bef2-cd53-4508-9fd9-b7613815d546 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:31:26.162276 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] deleting rbd volume volume-e3d7bef2-cd53-4508-9fd9-b7613815d546 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:31:26.164855 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:31:27.385983 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] deleting parent snapshot volume-40376eea-0c0f-488c-ae00-97b792b66c72.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:31:33.793268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] Created reservations ['a3ac329b-0eb0-4ff7-84e1-2a8b2895fca9', '60da3785-fbb0-470f-aa11-b2cb5c3355b8', 'b73b8929-e7ef-4d44-8f74-62d8f4b9ec36', '457c5157-da11-47f3-a4ec-54ee0d40f304'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:31:34.038306 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:34.151710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:34.228668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:31:34.238287 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] Deleted volume successfully. May 14 21:31:34.361726 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-824d6f4f-90da-41c4-883e-4bb9027ce954 tempest-TaggedBootDevicesTest_v242-387890351 None] Lock "b'cinder-40376eea-0c0f-488c-ae00-97b792b66c72-delete_volume'" released by "delete_volume" :: held 24.739s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:31:41.654372 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] Created reservations ['b6e56566-f0c5-4c73-81d9-ea94e9911e61', 'bf36cab1-ba3b-4b6e-a828-b6ac97b59e06', '392b2480-7847-47a2-8eb4-1d19a0169437', 'b6e869f4-63c4-424c-bf06-7d6452b938a3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:31:42.045851 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:42.247261 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:42.337932 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:31:42.346337 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] Deleted volume successfully. May 14 21:31:42.450044 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bf424383-e7f8-494e-b0b4-d2dc44963124 tempest-AttachVolumeNegativeTest-1286916119 None] Lock "b'cinder-e3d7bef2-cd53-4508-9fd9-b7613815d546-delete_volume'" released by "delete_volume" :: held 17.704s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:31:42.459947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] Lock "b'cinder-1ba2acd5-df14-446f-a4d1-701795380c04-delete_volume'" acquired by "delete_volume" :: waited 0.023s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:31:42.624228 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:44.085276 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:31:44.086696 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] Volume volume-1ba2acd5-df14-446f-a4d1-701795380c04 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:31:44.589953 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] deleting rbd volume volume-1ba2acd5-df14-446f-a4d1-701795380c04 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:31:48.881393 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:31:48.944153 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Flow 'volume_create_manager' (54416e35-6e0d-4b36-b406-009de14614b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:31:48.983007 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c828fb5c-7c0a-4cf7-86d0-2c527a93d503) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:49.017936 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c828fb5c-7c0a-4cf7-86d0-2c527a93d503) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:31:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1650315087',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=01597bbf-2f11-4ddd-bf70-6785c56639f3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6641b342c1d440aea451919f391bc5db',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:31:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:31:49Z,use_quota=True,user_id='5586a3a946a0490498c42cb71f39ce1e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:49.028941 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cacaf359-326d-443a-b3a3-c1619f654273) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:49.041433 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cacaf359-326d-443a-b3a3-c1619f654273) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:49.052229 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf323b29-3ad5-4d02-b590-2bcfdac8297a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:49.056706 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf323b29-3ad5-4d02-b590-2bcfdac8297a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '01597bbf-2f11-4ddd-bf70-6785c56639f3', 'volume_name': 'volume-01597bbf-2f11-4ddd-bf70-6785c56639f3', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:49.084941 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3b29c4ac-0c45-495c-a6ba-3eb6c89cbec9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:49.097632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3b29c4ac-0c45-495c-a6ba-3eb6c89cbec9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:49.105664 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9bfdd133-448d-4fab-977b-14730153aba1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:49.107141 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Volume 01597bbf-2f11-4ddd-bf70-6785c56639f3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-01597bbf-2f11-4ddd-bf70-6785c56639f3', 'volume_size': 1} May 14 21:31:49.107837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] creating volume 'volume-01597bbf-2f11-4ddd-bf70-6785c56639f3' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:31:49.108869 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:50.244108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:31:50.244727 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:31:50.249155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9bfdd133-448d-4fab-977b-14730153aba1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-01597bbf-2f11-4ddd-bf70-6785c56639f3', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:50.255426 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3baaed7-486c-4b0e-989c-a5315cd863a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:50.299189 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Terminate volume connection completed successfully. May 14 21:31:50.348209 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Remove volume export completed successfully. May 14 21:31:50.368049 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Volume volume-01597bbf-2f11-4ddd-bf70-6785c56639f3 (01597bbf-2f11-4ddd-bf70-6785c56639f3): created successfully May 14 21:31:50.373877 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3baaed7-486c-4b0e-989c-a5315cd863a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:50.392227 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Flow 'volume_create_manager' (54416e35-6e0d-4b36-b406-009de14614b1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:31:50.437192 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-d4a3485e-9088-48b6-ac8b-2122497b6eae-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:31:50.454240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:50.938670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:31:51.915492 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-d4a3485e-9088-48b6-ac8b-2122497b6eae {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:31:51.930009 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2830b250-a1b0-459f-bcaf-2aff9825121a tempest-AttachVolumeTestJSON-1845080071 None] Created volume successfully. May 14 21:31:52.624882 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-9679505f-6c9b-4840-a65b-11aacd39c6f6 req-ec7a7e81-2fe2-4a7e-8314-f41916b76f6b tempest-AttachVolumeTestJSON-1845080071 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:31:53.865655 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-9679505f-6c9b-4840-a65b-11aacd39c6f6 req-ec7a7e81-2fe2-4a7e-8314-f41916b76f6b tempest-AttachVolumeTestJSON-1845080071 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.241s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:31:53.867690 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-9679505f-6c9b-4840-a65b-11aacd39c6f6 req-ec7a7e81-2fe2-4a7e-8314-f41916b76f6b tempest-AttachVolumeTestJSON-1845080071 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-01597bbf-2f11-4ddd-bf70-6785c56639f3', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '01597bbf-2f11-4ddd-bf70-6785c56639f3', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:31:53.998972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-9679505f-6c9b-4840-a65b-11aacd39c6f6 req-ec7a7e81-2fe2-4a7e-8314-f41916b76f6b tempest-AttachVolumeTestJSON-1845080071 None] Connection info returned from driver {'name': 'volumes/volume-01597bbf-2f11-4ddd-bf70-6785c56639f3', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '01597bbf-2f11-4ddd-bf70-6785c56639f3', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a35f5f27-323c-4965-9fba-f295268cdbee'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:31:54.165248 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-9679505f-6c9b-4840-a65b-11aacd39c6f6 req-ec7a7e81-2fe2-4a7e-8314-f41916b76f6b tempest-AttachVolumeTestJSON-1845080071 None] attachment_update completed successfully. May 14 21:31:54.999420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] Created reservations ['8dcda541-bd6f-4cf9-8145-c02b56c263fb', 'c2a8898a-a2d5-414f-8acf-8c034d498d81', '53729d35-18cd-4baa-a1e2-938705005756', '3024f40e-6da2-4a6c-9208-0aff7e828f51'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:31:55.192748 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:55.259986 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:55.336464 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:31:55.341476 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] Deleted volume successfully. May 14 21:31:55.479193 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-2158b766-9467-4cf2-9289-3a91b407ad66 tempest-AttachVolumeNegativeTest-1286916119 None] Lock "b'cinder-1ba2acd5-df14-446f-a4d1-701795380c04-delete_volume'" released by "delete_volume" :: held 13.019s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:31:58.394779 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:31:58.441291 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Flow 'volume_create_manager' (a1f8b9af-f007-4a74-b29d-9dd04ed17cbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:31:58.448462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (900054c6-6446-40c6-bd3d-e4923727ef10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:58.494020 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (900054c6-6446-40c6-bd3d-e4923727ef10) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:31:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1358981156',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=db8974a4-ada7-45fe-acdd-6123c7b72686,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6641b342c1d440aea451919f391bc5db',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:31:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:31:58Z,use_quota=True,user_id='5586a3a946a0490498c42cb71f39ce1e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:58.508828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3390d388-0431-4f02-a18f-8483ec09e7ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:58.518008 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3390d388-0431-4f02-a18f-8483ec09e7ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:58.534509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4e4e5d15-e4b2-4f16-8d10-a5d47b563e0d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:58.539775 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4e4e5d15-e4b2-4f16-8d10-a5d47b563e0d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'db8974a4-ada7-45fe-acdd-6123c7b72686', 'volume_name': 'volume-db8974a4-ada7-45fe-acdd-6123c7b72686', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:58.549926 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (48a16c0c-e515-432b-b2be-b592d7665ee7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:58.574052 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (48a16c0c-e515-432b-b2be-b592d7665ee7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:58.580977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (99333599-3cc1-4e34-b1f3-36ff03744eeb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:31:58.582697 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Volume db8974a4-ada7-45fe-acdd-6123c7b72686: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-db8974a4-ada7-45fe-acdd-6123c7b72686', 'volume_size': 1} May 14 21:31:58.583111 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] creating volume 'volume-db8974a4-ada7-45fe-acdd-6123c7b72686' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:31:58.583945 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:31:59.969825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:31:59.969825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:31:59.983272 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (99333599-3cc1-4e34-b1f3-36ff03744eeb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-db8974a4-ada7-45fe-acdd-6123c7b72686', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:31:59.991349 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6a42b84a-0ca3-4000-af10-b4b263833ee2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:00.144811 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Volume volume-db8974a4-ada7-45fe-acdd-6123c7b72686 (db8974a4-ada7-45fe-acdd-6123c7b72686): created successfully May 14 21:32:00.145953 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6a42b84a-0ca3-4000-af10-b4b263833ee2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:00.156098 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Flow 'volume_create_manager' (a1f8b9af-f007-4a74-b29d-9dd04ed17cbf) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:32:00.277844 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-529761ba-0130-4646-b959-65d3487ba279 tempest-AttachVolumeTestJSON-1845080071 None] Created volume successfully. May 14 21:32:00.303196 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-88d24d17-99e2-4724-bf09-602bcf131253 req-719d77f7-ac39-4b43-b4d6-e56b49616078 tempest-AttachVolumeShelveTestJSON-565426114 None] No connector for attachment e006ecb5-0992-48bd-8ee4-df3e88f3b359; skipping storage backend terminate_connection call. {{(pid=123870) _connection_terminate /opt/stack/cinder/cinder/volume/manager.py:4910}} May 14 21:32:00.303747 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-88d24d17-99e2-4724-bf09-602bcf131253 req-719d77f7-ac39-4b43-b4d6-e56b49616078 tempest-AttachVolumeShelveTestJSON-565426114 None] Deleting attachment e006ecb5-0992-48bd-8ee4-df3e88f3b359. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:32:00.884401 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:32:00.891757 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5a75b09-1e06-4293-883f-ba15683bb6e6 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-d4a3485e-9088-48b6-ac8b-2122497b6eae-delete_volume'" released by "delete_volume" :: held 10.455s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:32:02.982704 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-2f05bdbd-ad45-4d9c-b983-436376046ada req-342f90c0-9786-4a40-beb2-cf3db7d4de63 tempest-AttachVolumeTestJSON-1845080071 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:32:03.671853 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:32:03.673024 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:03.802671 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:03.888237 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:32:04.439036 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-2f05bdbd-ad45-4d9c-b983-436376046ada req-342f90c0-9786-4a40-beb2-cf3db7d4de63 tempest-AttachVolumeTestJSON-1845080071 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.456s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:32:04.440451 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-2f05bdbd-ad45-4d9c-b983-436376046ada req-342f90c0-9786-4a40-beb2-cf3db7d4de63 tempest-AttachVolumeTestJSON-1845080071 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-db8974a4-ada7-45fe-acdd-6123c7b72686', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'db8974a4-ada7-45fe-acdd-6123c7b72686', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:32:04.560129 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-2f05bdbd-ad45-4d9c-b983-436376046ada req-342f90c0-9786-4a40-beb2-cf3db7d4de63 tempest-AttachVolumeTestJSON-1845080071 None] Connection info returned from driver {'name': 'volumes/volume-db8974a4-ada7-45fe-acdd-6123c7b72686', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'db8974a4-ada7-45fe-acdd-6123c7b72686', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '455cee83-0c65-4e1a-b1f6-c2f22cb12b34'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:32:04.764174 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-2f05bdbd-ad45-4d9c-b983-436376046ada req-342f90c0-9786-4a40-beb2-cf3db7d4de63 tempest-AttachVolumeTestJSON-1845080071 None] attachment_update completed successfully. May 14 21:32:09.862379 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:32:09.906610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Flow 'volume_create_manager' (60393bc2-2298-40d6-95f5-bc71f09fe499) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:32:09.938578 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9a3e3fc1-3cb9-4b61-b55d-9efd4d7d47c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:09.974619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9a3e3fc1-3cb9-4b61-b55d-9efd4d7d47c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:32:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotsTestJSON-volume-1986164432',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6ab899de-8a3b-4d6c-85de-5c4440b09dfb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2a1393fd6fd94533b439ca5f156d94ee',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:32:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:32:10Z,use_quota=True,user_id='0e40446fedca416883f7321a3de390bf',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:09.991517 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (21407ec4-548c-43b2-9806-b19785dcabe2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:09.997802 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (21407ec4-548c-43b2-9806-b19785dcabe2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:10.016829 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (27b99ef5-33d6-4276-a0c2-11f79728ed0f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:10.021500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (27b99ef5-33d6-4276-a0c2-11f79728ed0f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6ab899de-8a3b-4d6c-85de-5c4440b09dfb', 'volume_name': 'volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:10.041100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82bf7d0d-a9d7-4400-83aa-3982b854deab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:10.041939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82bf7d0d-a9d7-4400-83aa-3982b854deab) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:10.050607 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f971b7ee-b1c8-461f-a41c-7b8284329d46) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:10.052768 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Volume 6ab899de-8a3b-4d6c-85de-5c4440b09dfb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb', 'volume_size': 1} May 14 21:32:10.053727 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] creating volume 'volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:32:10.054997 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:10.226619 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-52dbf432-897f-4870-9f23-ec0e61a61b67 req-6a5655e8-0135-4d2e-ac62-e84198ee8307 tempest-AttachVolumeTestJSON-1845080071 None] Terminate volume connection completed successfully. May 14 21:32:10.227113 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-52dbf432-897f-4870-9f23-ec0e61a61b67 req-6a5655e8-0135-4d2e-ac62-e84198ee8307 tempest-AttachVolumeTestJSON-1845080071 None] Deleting attachment a35f5f27-323c-4965-9fba-f295268cdbee. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:32:11.356950 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:32:11.358436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:32:11.370462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f971b7ee-b1c8-461f-a41c-7b8284329d46) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:11.382301 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a2d390d-c9d5-44e2-a32f-5407ca3aa116) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:11.502027 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Volume volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb (6ab899de-8a3b-4d6c-85de-5c4440b09dfb): created successfully May 14 21:32:11.509245 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a2d390d-c9d5-44e2-a32f-5407ca3aa116) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:11.540019 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Flow 'volume_create_manager' (60393bc2-2298-40d6-95f5-bc71f09fe499) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:32:11.655642 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-803196bf-e9c0-406f-b5b9-16ea5162557d req-42119326-2275-4271-8f6b-401adcbd878a tempest-VolumesSnapshotsTestJSON-2001944209 None] Created volume successfully. May 14 21:32:13.259368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-d1a504f3-e7e9-4a90-b0e0-b2dda9972646 req-f5e46c63-eeca-4ad9-b6d0-f3427368ffc1 tempest-VolumesSnapshotsTestJSON-2001944209 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:13.609179 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d5ef0ed3-27f2-4678-bd03-1ba98d4a066a req-3b42c05d-20db-4f69-ab9d-831ab08a020b tempest-AttachVolumeTestJSON-1845080071 None] Terminate volume connection completed successfully. May 14 21:32:13.610730 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d5ef0ed3-27f2-4678-bd03-1ba98d4a066a req-3b42c05d-20db-4f69-ab9d-831ab08a020b tempest-AttachVolumeTestJSON-1845080071 None] Deleting attachment 455cee83-0c65-4e1a-b1f6-c2f22cb12b34. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:32:17.461294 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d1a504f3-e7e9-4a90-b0e0-b2dda9972646 req-f5e46c63-eeca-4ad9-b6d0-f3427368ffc1 tempest-VolumesSnapshotsTestJSON-2001944209 None] Create snapshot completed successfully May 14 21:32:19.202028 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-3fd6dc39-f8c0-45ac-9f66-bf26df7ba56e req-a6386ee8-4ac0-46ed-aa94-764084fea96a tempest-VolumesSnapshotsTestJSON-2001944209 None] Lock "b'cinder-b37d9e4b-d59e-445f-8880-97f68de7f7ce-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:32:19.204371 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-3fd6dc39-f8c0-45ac-9f66-bf26df7ba56e req-a6386ee8-4ac0-46ed-aa94-764084fea96a tempest-VolumesSnapshotsTestJSON-2001944209 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:24.410986 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [req-3fd6dc39-f8c0-45ac-9f66-bf26df7ba56e req-a6386ee8-4ac0-46ed-aa94-764084fea96a tempest-VolumesSnapshotsTestJSON-2001944209 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota. May 14 21:32:24.523952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-3fd6dc39-f8c0-45ac-9f66-bf26df7ba56e req-a6386ee8-4ac0-46ed-aa94-764084fea96a tempest-VolumesSnapshotsTestJSON-2001944209 None] Created reservations ['298489ed-3541-4514-ba00-5365014b0d85', '76783208-11c4-4deb-af83-588391101296', '06a34db0-7d68-4cf6-a08d-e70f07924b5a', '3fdd4b7c-cc89-4f3d-a210-48df041fc7ff'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:32:24.924050 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-3fd6dc39-f8c0-45ac-9f66-bf26df7ba56e req-a6386ee8-4ac0-46ed-aa94-764084fea96a tempest-VolumesSnapshotsTestJSON-2001944209 None] Delete snapshot completed successfully. May 14 21:32:24.934340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-3fd6dc39-f8c0-45ac-9f66-bf26df7ba56e req-a6386ee8-4ac0-46ed-aa94-764084fea96a tempest-VolumesSnapshotsTestJSON-2001944209 None] Lock "b'cinder-b37d9e4b-d59e-445f-8880-97f68de7f7ce-delete_snapshot'" released by "delete_snapshot" :: held 5.732s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:32:25.979030 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] Lock "b'cinder-6ab899de-8a3b-4d6c-85de-5c4440b09dfb-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:32:26.099132 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:26.374845 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:32:26.375861 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] Volume volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:32:26.592527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] deleting rbd volume volume-6ab899de-8a3b-4d6c-85de-5c4440b09dfb {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:32:29.006784 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] Lock "b'cinder-db8974a4-ada7-45fe-acdd-6123c7b72686-delete_volume'" acquired by "delete_volume" :: waited 0.041s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:32:29.161721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:29.443825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:32:29.444316 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] Volume volume-db8974a4-ada7-45fe-acdd-6123c7b72686 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:32:29.743534 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] deleting rbd volume volume-db8974a4-ada7-45fe-acdd-6123c7b72686 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:32:35.911616 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:32:35.996776 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (e48a47b2-1255-4fb3-9449-56d645701fb2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:32:36.023808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (589b0638-4103-4dd9-8847-03678ea46a1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:36.029798 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] Created reservations ['ef2218c7-8b18-424c-a5ba-a2b9be3b101d', 'bb2005ce-1efe-4e6d-88d1-4288ad1a02d2', '5210e5f1-89b4-4641-b40e-f1088ff0a8d4', 'a762f86a-6fb5-4fdc-a09b-da559b6124b1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:32:36.058615 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (589b0638-4103-4dd9-8847-03678ea46a1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:32:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1112096046',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cd3708f6-e860-49e0-862f-98ce06d603c8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='162610ac2697486d8a05f41713cc5ba2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:32:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:32:36Z,use_quota=True,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:36.064555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (28205278-d546-43c0-bf4c-d85cdff69662) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:36.069266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (28205278-d546-43c0-bf4c-d85cdff69662) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:36.075841 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ea05b541-01d7-4a48-8905-d6893e442165) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:36.202785 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ea05b541-01d7-4a48-8905-d6893e442165) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'cd3708f6-e860-49e0-862f-98ce06d603c8', 'volume_name': 'volume-cd3708f6-e860-49e0-862f-98ce06d603c8', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:36.209839 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2783def3-a122-46af-be45-340d3c29f587) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:36.215255 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2783def3-a122-46af-be45-340d3c29f587) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:36.220430 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9e163350-c4e9-41b0-8c90-c5698cfa79fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:36.221611 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume cd3708f6-e860-49e0-862f-98ce06d603c8: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-cd3708f6-e860-49e0-862f-98ce06d603c8', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:32:36.222196 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Cloning cd3708f6-e860-49e0-862f-98ce06d603c8 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:32:36.248444 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.023s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:32:36.318286 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 32, 36, 254123)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:32:36.319235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:32:36.319860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:32:36.327220 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.079s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:32:36.328759 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:32:36.335821 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 32, 36, 333416)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:32:36.336541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:32:36.337740 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:32:36.352886 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:36.666414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] creating snapshot='volume-cd3708f6-e860-49e0-862f-98ce06d603c8.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:32:39.038896 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-cd3708f6-e860-49e0-862f-98ce06d603c8.clone_snap' to 'volume-cd3708f6-e860-49e0-862f-98ce06d603c8' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:32:39.043017 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:39.180625 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:39.254810 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:32:39.268085 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] Deleted volume successfully. May 14 21:32:39.377051 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-88f8c7cc-39a9-47f7-ab02-42adae5040dd req-bb4676d1-b30a-43c0-8d47-de0abb6e8c60 tempest-VolumesSnapshotsTestJSON-2001944209 None] Lock "b'cinder-6ab899de-8a3b-4d6c-85de-5c4440b09dfb-delete_volume'" released by "delete_volume" :: held 13.398s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:32:39.459863 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] Created reservations ['3d55cbeb-fe07-468e-8a3a-129f2ffe3d5d', '653950af-5795-42c6-a019-512d1f357903', '85082038-384e-4cc8-a8fd-cf0b5674442e', 'cebb1810-8927-43c8-b6b4-cfc12b6d61a3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:32:39.758012 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:39.830370 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:39.906641 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:32:39.911022 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] Deleted volume successfully. May 14 21:32:40.141077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-be0afa7b-24b8-4ad5-974e-2eff576a870b tempest-AttachVolumeTestJSON-1845080071 None] Lock "b'cinder-db8974a4-ada7-45fe-acdd-6123c7b72686-delete_volume'" released by "delete_volume" :: held 11.134s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:32:40.397985 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] Lock "b'cinder-01597bbf-2f11-4ddd-bf70-6785c56639f3-delete_volume'" acquired by "delete_volume" :: waited 0.021s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:32:40.557809 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:40.866097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:32:40.878871 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] Volume volume-01597bbf-2f11-4ddd-bf70-6785c56639f3 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:32:41.311220 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] deleting rbd volume volume-01597bbf-2f11-4ddd-bf70-6785c56639f3 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:32:41.793683 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:32:41.795159 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:32:41.795700 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:32:42.549413 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:32:42.556698 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Marking volume cd3708f6-e860-49e0-862f-98ce06d603c8 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:32:42.626980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to cd3708f6-e860-49e0-862f-98ce06d603c8. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:32:42.626980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Creating volume glance metadata for volume cd3708f6-e860-49e0-862f-98ce06d603c8 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:32:42.881455 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9e163350-c4e9-41b0-8c90-c5698cfa79fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cd3708f6-e860-49e0-862f-98ce06d603c8', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:42.886155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (95136171-2dc5-44ae-8811-8e4c99d3e619) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:32:43.087110 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-cd3708f6-e860-49e0-862f-98ce06d603c8 (cd3708f6-e860-49e0-862f-98ce06d603c8): created successfully May 14 21:32:43.092484 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (95136171-2dc5-44ae-8811-8e4c99d3e619) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:32:43.103420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (e48a47b2-1255-4fb3-9449-56d645701fb2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:32:43.226755 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-95c01107-e8ff-404a-b9d3-6a2a91da9f48 tempest-VolumesSnapshotTestJSON-1533504236 None] Created volume successfully. May 14 21:32:44.864338 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-7319dc25-4ec5-41f9-8d8a-4fd776325416 req-0376f14b-4c12-44dd-b6fd-e93a6758a223 tempest-VolumesSnapshotTestJSON-1533504236 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:32:45.668139 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-7319dc25-4ec5-41f9-8d8a-4fd776325416 req-0376f14b-4c12-44dd-b6fd-e93a6758a223 tempest-VolumesSnapshotTestJSON-1533504236 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.803s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:32:45.668974 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-7319dc25-4ec5-41f9-8d8a-4fd776325416 req-0376f14b-4c12-44dd-b6fd-e93a6758a223 tempest-VolumesSnapshotTestJSON-1533504236 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-cd3708f6-e860-49e0-862f-98ce06d603c8', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'cd3708f6-e860-49e0-862f-98ce06d603c8', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:32:45.819447 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-7319dc25-4ec5-41f9-8d8a-4fd776325416 req-0376f14b-4c12-44dd-b6fd-e93a6758a223 tempest-VolumesSnapshotTestJSON-1533504236 None] Connection info returned from driver {'name': 'volumes/volume-cd3708f6-e860-49e0-862f-98ce06d603c8', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'cd3708f6-e860-49e0-862f-98ce06d603c8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '35e9eaa5-ae44-4a00-b9d8-896444bac6d6'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:32:46.011147 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-7319dc25-4ec5-41f9-8d8a-4fd776325416 req-0376f14b-4c12-44dd-b6fd-e93a6758a223 tempest-VolumesSnapshotTestJSON-1533504236 None] attachment_update completed successfully. May 14 21:32:49.821632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] Created reservations ['379bcc44-4222-4c02-8326-10370e28e1cd', 'afaae771-5446-4dea-a44a-7f3de5911d1a', 'c1a7278b-5ee6-44bf-aeac-d987d6597788', 'd178ca7b-f24f-4fcd-ae02-0a0b20b9bd63'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:32:50.078778 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-20a14506-21f5-4408-9dba-03d9ff5db7a4 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:50.451957 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:50.493748 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:50.522735 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:32:50.527336 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] Deleted volume successfully. May 14 21:32:50.676197 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8cd9016a-26ab-4536-9729-a71c11ab0634 tempest-AttachVolumeTestJSON-1845080071 None] Lock "b'cinder-01597bbf-2f11-4ddd-bf70-6785c56639f3-delete_volume'" released by "delete_volume" :: held 10.279s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:32:50.694995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] Lock "b'cinder-fe4b9b45-3bd4-47a6-bd85-901d2fa90799-delete_volume'" acquired by "delete_volume" :: waited 0.028s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:32:50.869368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:51.334891 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:32:51.335141 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] Volume volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:32:51.718180 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] deleting rbd volume volume-fe4b9b45-3bd4-47a6-bd85-901d2fa90799 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:32:53.123806 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-20a14506-21f5-4408-9dba-03d9ff5db7a4 tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:32:54.662411 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6625b87d-f608-4628-86d2-c25a63918d7e tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:32:58.050352 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6625b87d-f608-4628-86d2-c25a63918d7e tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:32:59.497010 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f97f936c-e5e8-4018-98c3-8e7b4b39b7eb tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:00.118480 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] Created reservations ['b406b045-54aa-4da3-86e0-61eeb66fdc45', '37ae6137-5320-411f-b069-0b3c49a5dd92', 'd68dc97f-d098-4b78-8d36-0dc31e4a6051', '2dc0233d-a411-46f2-abd2-ac92bc25387e'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:33:00.421406 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:00.502105 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:00.664755 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:33:00.670786 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] Deleted volume successfully. May 14 21:33:00.759047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b78c83e6-db0f-4369-b189-3a5074bec542 tempest-AttachVolumeTestJSON-1845080071 None] Lock "b'cinder-fe4b9b45-3bd4-47a6-bd85-901d2fa90799-delete_volume'" released by "delete_volume" :: held 10.070s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:33:02.235774 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f97f936c-e5e8-4018-98c3-8e7b4b39b7eb tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:33:02.600126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-198d2c25-b3fb-409b-8369-f04fd356de68 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-3a031f41-f089-4b5e-bbd2-eceba3c5563c-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.017s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:33:02.602037 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-198d2c25-b3fb-409b-8369-f04fd356de68 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:03.668534 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:33:03.670625 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:03.748365 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:03.797791 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:33:14.514721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] Lock "b'cinder-11d02ad8-e260-47a5-8195-f9c8f8b8ea85-delete_volume'" acquired by "delete_volume" :: waited 0.024s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:33:14.691983 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:15.007042 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:33:15.007970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] Volume volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:33:15.301764 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] deleting rbd volume volume-11d02ad8-e260-47a5-8195-f9c8f8b8ea85 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:33:23.255544 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] Created reservations ['34364246-587a-4f23-95a5-ffa6aebbd610', 'c5aa519f-ede5-48a6-ac8e-3841fe450912', 'a0b2c5e1-28d8-45e1-90c8-454fef59028a', '86d3d850-64a1-45d0-bde5-f67915beb977'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:33:23.489496 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:23.623090 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:23.658732 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:33:23.663392 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] Deleted volume successfully. May 14 21:33:23.740254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-00955c2d-686a-4f83-ac11-ec258db33c2f tempest-AttachVolumeShelveTestJSON-565426114 None] Lock "b'cinder-11d02ad8-e260-47a5-8195-f9c8f8b8ea85-delete_volume'" released by "delete_volume" :: held 9.226s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:33:24.882845 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] Lock "b'cinder-d520492e-c3a1-42a9-a065-6b741defc4c0-delete_volume'" acquired by "delete_volume" :: waited 0.021s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:33:25.046551 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:25.353284 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:33:25.354045 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] Volume volume-d520492e-c3a1-42a9-a065-6b741defc4c0 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:33:25.562421 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] deleting rbd volume volume-d520492e-c3a1-42a9-a065-6b741defc4c0 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:33:33.654336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] Created reservations ['22da5d0a-79b2-4109-8231-84df08a48665', '56ab9e21-9bfe-4208-b5c9-45672fa013e2', 'b3ab8f7f-b07c-4b4c-8bdc-7541371f3ccf', '3ad8f2ca-e013-4d64-85c5-c3d734587cd0'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:33:33.947431 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:33.994430 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:34.046203 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:33:34.055625 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] Deleted volume successfully. May 14 21:33:34.213001 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a3a6c1a5-3274-451b-b0b7-2cb737067799 tempest-AttachVolumeShelveTestJSON-565426114 None] Lock "b'cinder-d520492e-c3a1-42a9-a065-6b741defc4c0-delete_volume'" released by "delete_volume" :: held 9.330s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:33:34.350166 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-198d2c25-b3fb-409b-8369-f04fd356de68 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['64abe440-57f2-4444-97e7-4ef60c93c53b', '9f2bad12-2f68-46c0-9e24-a582d02289c4', '26061de6-0947-474c-bc98-228d7f06c227', 'e5760922-7d61-4b73-b7dc-d207ba62f388'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:33:34.838409 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-198d2c25-b3fb-409b-8369-f04fd356de68 tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:33:34.849425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-198d2c25-b3fb-409b-8369-f04fd356de68 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-3a031f41-f089-4b5e-bbd2-eceba3c5563c-delete_snapshot'" released by "delete_snapshot" :: held 32.250s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:33:35.095513 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-523ec255-4279-46ab-841f-f04a581be8ee tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-ac7382e9-7a99-462a-b458-2e4b0b6b927c-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.020s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:33:35.097378 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-523ec255-4279-46ab-841f-f04a581be8ee tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:39.748898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:33:39.795760 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Flow 'volume_create_manager' (18b6a1da-02c9-4a62-a881-3e193e37729f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:33:39.806837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (115694c3-f8ec-41bd-8f5f-c828644871f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:33:39.823630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (115694c3-f8ec-41bd-8f5f-c828644871f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:33:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-100050685',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ba6689c8-5731-4271-ab38-0b3532441e18,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='59c0f4df562340e5bb096ef2eed9239e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:33:40Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:33:40Z,use_quota=True,user_id='cbb8c5cf2d144e3791171eb3d6f83eca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:33:39.831802 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c6499b1b-8edd-4a28-8146-368e1ff22794) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:33:39.842452 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c6499b1b-8edd-4a28-8146-368e1ff22794) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:33:39.852997 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (14a231ed-b397-4ff6-8942-f32c6ec7207f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:33:39.858646 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (14a231ed-b397-4ff6-8942-f32c6ec7207f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ba6689c8-5731-4271-ab38-0b3532441e18', 'volume_name': 'volume-ba6689c8-5731-4271-ab38-0b3532441e18', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:33:39.871141 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d2e3981b-b2bf-456f-b7f1-9b6dae3ed09a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:33:39.876550 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d2e3981b-b2bf-456f-b7f1-9b6dae3ed09a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:33:39.885446 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6c7e05ca-2798-4bfd-b9ce-751bbb214236) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:33:39.888763 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Volume ba6689c8-5731-4271-ab38-0b3532441e18: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ba6689c8-5731-4271-ab38-0b3532441e18', 'volume_size': 1} May 14 21:33:39.896313 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] creating volume 'volume-ba6689c8-5731-4271-ab38-0b3532441e18' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:33:39.897404 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:33:40.980520 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:33:40.981084 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:33:40.985320 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6c7e05ca-2798-4bfd-b9ce-751bbb214236) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ba6689c8-5731-4271-ab38-0b3532441e18', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:33:40.994079 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (604cc6a7-2e95-43f0-8a51-b88257784f77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:33:41.130236 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Volume volume-ba6689c8-5731-4271-ab38-0b3532441e18 (ba6689c8-5731-4271-ab38-0b3532441e18): created successfully May 14 21:33:41.130236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (604cc6a7-2e95-43f0-8a51-b88257784f77) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:33:41.145563 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Flow 'volume_create_manager' (18b6a1da-02c9-4a62-a881-3e193e37729f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:33:41.250623 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bf426a6d-65e2-429b-8ee8-d39870a63e60 tempest-ServerRescueNegativeTestJSON-879335362 None] Created volume successfully. May 14 21:34:03.664374 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:34:03.665220 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:03.714175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:03.746780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:34:04.753419 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-dfc65553-f86f-46f1-ab2f-003cf718b34e tempest-GroupSnapshotsV319Test-1073273508 None] Group tempest-GroupSnapshotsV319Test-Group-1022341853: creating May 14 21:34:04.870893 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-dfc65553-f86f-46f1-ab2f-003cf718b34e tempest-GroupSnapshotsV319Test-1073273508 None] Group tempest-GroupSnapshotsV319Test-Group-1022341853: created successfully May 14 21:34:04.884025 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-dfc65553-f86f-46f1-ab2f-003cf718b34e tempest-GroupSnapshotsV319Test-1073273508 None] Create group completed successfully. May 14 21:34:06.729023 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-523ec255-4279-46ab-841f-f04a581be8ee tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['d95b5a7f-2aa2-4e20-8ac1-30996bd9db33', 'cdb3502a-92de-4ac2-b585-9ee5cec19a52', '9199c70b-8136-43b5-9e78-d074329be585', 'e276333d-fffa-4a83-af85-8ca7e441163c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:07.058978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:34:07.080794 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Flow 'volume_create_manager' (d004ee23-6f45-48f9-90fa-0693f0c3f64e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:34:07.087636 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7935c54f-2956-4d47-ae1e-46756e71d14e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.093095 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-523ec255-4279-46ab-841f-f04a581be8ee tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:34:07.104550 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-523ec255-4279-46ab-841f-f04a581be8ee tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-ac7382e9-7a99-462a-b458-2e4b0b6b927c-delete_snapshot'" released by "delete_snapshot" :: held 32.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:34:07.110878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7935c54f-2956-4d47-ae1e-46756e71d14e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:34:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-2099404400',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9ee16695-14bc-482f-8ed9-e3a38eb3ecc6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='59c0f4df562340e5bb096ef2eed9239e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:34:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:34:07Z,use_quota=True,user_id='cbb8c5cf2d144e3791171eb3d6f83eca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.116499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7df6bbc9-68b6-4f9e-bf76-6fd37d0ca84e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.121749 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7df6bbc9-68b6-4f9e-bf76-6fd37d0ca84e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.127139 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (95f7dc59-422d-4252-8052-f90e4c0037e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.131654 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (95f7dc59-422d-4252-8052-f90e4c0037e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'volume_name': 'volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.137474 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e47e2f21-12ec-45e5-9dcd-e6d9eda3a819) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.141500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e47e2f21-12ec-45e5-9dcd-e6d9eda3a819) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.146511 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c82c67b9-dddb-4adb-be9d-9b90a27b1b4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.147629 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Volume 9ee16695-14bc-482f-8ed9-e3a38eb3ecc6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'volume_size': 1} May 14 21:34:07.148028 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] creating volume 'volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:34:07.149153 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:07.193620 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:34:07.256470 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Flow 'volume_create_manager' (54a3adcc-3658-40b2-9fbc-e7af19992536) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:34:07.270652 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (51ca625b-bc85-4b23-8421-0eea8bf75784) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.293218 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (51ca625b-bc85-4b23-8421-0eea8bf75784) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:34:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsV319Test-Volume-1627987623',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=a92b0565-7c4b-4ee8-b3db-1d4140717a9a,host='ncdv-master-1@ceph#ceph',id=7aa8c938-3841-4361-9be9-5d0fb3fd707f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bc31f3fccb75482f9d0fe81ba4be186a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:34:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:34:07Z,use_quota=True,user_id='add685ef1a92432da84cdf6520d136d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(35d8b183-b46c-45e1-8d91-ce2cd48a4b73),volume_type_id=35d8b183-b46c-45e1-8d91-ce2cd48a4b73)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.299690 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (dec6b8b7-d43e-406a-8422-844e97200a0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.304319 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (dec6b8b7-d43e-406a-8422-844e97200a0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.310026 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (595ea356-a442-4a1b-a890-9426454ffa10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.314136 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (595ea356-a442-4a1b-a890-9426454ffa10) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7aa8c938-3841-4361-9be9-5d0fb3fd707f', 'volume_name': 'volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.319209 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cb9559cb-cdc2-40e9-957d-6631c826a1b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.324612 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cb9559cb-cdc2-40e9-957d-6631c826a1b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:07.329398 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dc6c4e67-0c9e-4c57-8656-cc1880d99525) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:07.330747 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Volume 7aa8c938-3841-4361-9be9-5d0fb3fd707f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f', 'volume_size': 1} May 14 21:34:07.331157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] creating volume 'volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:34:07.331854 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:07.780455 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-1a0fbeff-fb79-40a4-82a6-4b8636cb3aa8 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-011f772f-07ff-4ec2-be7f-0280aa275ad5-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:34:07.781981 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1a0fbeff-fb79-40a4-82a6-4b8636cb3aa8 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:08.381714 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:34:08.382100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:34:08.386482 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c82c67b9-dddb-4adb-be9d-9b90a27b1b4a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:08.391279 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (93e3ad29-ef10-441e-85c9-4bd7d331d356) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:08.495250 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Volume volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6 (9ee16695-14bc-482f-8ed9-e3a38eb3ecc6): created successfully May 14 21:34:08.499081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (93e3ad29-ef10-441e-85c9-4bd7d331d356) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:08.505237 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Flow 'volume_create_manager' (d004ee23-6f45-48f9-90fa-0693f0c3f64e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:34:08.568178 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0dee8afd-4f20-403c-a929-cf932d757a6c tempest-ServerRescueNegativeTestJSON-879335362 None] Created volume successfully. May 14 21:34:08.631346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:34:08.631346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:34:08.632504 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dc6c4e67-0c9e-4c57-8656-cc1880d99525) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:08.637841 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e615c046-faa2-4879-9be7-1ce037f76ea4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:08.720290 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Volume volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f (7aa8c938-3841-4361-9be9-5d0fb3fd707f): created successfully May 14 21:34:08.724454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e615c046-faa2-4879-9be7-1ce037f76ea4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:08.731427 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Flow 'volume_create_manager' (54a3adcc-3658-40b2-9fbc-e7af19992536) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:34:08.800509 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ebec3eb3-0723-454e-9ed4-a821d1660d46 tempest-GroupSnapshotsV319Test-1073273508 None] Created volume successfully. May 14 21:34:09.603332 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-484da421-3fa0-416a-9608-8f65af27c2b5 tempest-GroupSnapshotsV319Test-1073273508 None] GroupSnapshot 5bfefcf3-8078-48d7-9da0-435e5a5ef9fd: creating. May 14 21:34:09.641396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-484da421-3fa0-416a-9608-8f65af27c2b5 tempest-GroupSnapshotsV319Test-1073273508 None] Group snapshot 5bfefcf3-8078-48d7-9da0-435e5a5ef9fd: creating. {{(pid=123870) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4078}} May 14 21:34:09.660541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-484da421-3fa0-416a-9608-8f65af27c2b5 tempest-GroupSnapshotsV319Test-1073273508 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:12.895873 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-484da421-3fa0-416a-9608-8f65af27c2b5 tempest-GroupSnapshotsV319Test-1073273508 None] group_snapshot 5bfefcf3-8078-48d7-9da0-435e5a5ef9fd: created successfully May 14 21:34:14.971514 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] group_snapshot 5bfefcf3-8078-48d7-9da0-435e5a5ef9fd: deleting May 14 21:34:15.004980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] group_snapshot 5bfefcf3-8078-48d7-9da0-435e5a5ef9fd: deleting {{(pid=123870) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4253}} May 14 21:34:15.063493 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:19.645934 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsV319Test-volume-type-243306976 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsV319Test-volume-type-243306976, it is now deprecated. Please use the default quota class for default quota. May 14 21:34:19.646819 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsV319Test-volume-type-243306976 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsV319Test-volume-type-243306976, it is now deprecated. Please use the default quota class for default quota. May 14 21:34:19.768223 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] Created reservations ['b2beec18-fc79-41c8-b8be-68f605ab057e', 'f42cf8d1-e694-481e-9975-dfadfdad4791', 'e89d5b7f-3b77-40ca-8312-396ce55527d8', '22d98a71-988f-4d0d-bb90-d57b614f4ba8'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:20.065084 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6ce1ba62-d107-41cd-9d36-aa0c85e67241 tempest-GroupSnapshotsV319Test-1073273508 None] group_snapshot 5bfefcf3-8078-48d7-9da0-435e5a5ef9fd: deleted successfully May 14 21:34:21.153948 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:21.430629 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:34:21.431815 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] Volume volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:34:21.665316 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] deleting rbd volume volume-7aa8c938-3841-4361-9be9-5d0fb3fd707f {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:34:28.431940 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] Created reservations ['8e49f785-1e9f-43c2-be53-3b74374d19b7'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:28.472081 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsV319Test-volume-type-243306976 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsV319Test-volume-type-243306976, it is now deprecated. Please use the default quota class for default quota. May 14 21:34:28.542971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] Created reservations ['34492d06-3904-4c12-9b6c-164a095bf763', 'aeb7d7d2-dab1-479c-8825-66ba286498e5', '9b4c8219-09f3-48f0-a7da-0dd9c93484ac', '8f68d44c-4a19-4e24-a8fc-b6fa47748fcc'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:29.042009 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:29.107969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:29.178477 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:34:29.199450 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e8ba53da-ce64-4f77-918e-46cb7bea672d tempest-GroupSnapshotsV319Test-1073273508 None] Delete group completed successfully. May 14 21:34:38.667261 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:34:38.689663 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Flow 'volume_create_manager' (eba2500b-acb2-4994-93aa-568074bfe698) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:34:38.697630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5d5747a9-7c31-4dcb-8059-6c569db0aab0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:38.716863 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5d5747a9-7c31-4dcb-8059-6c569db0aab0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:34:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotsActionsTest-Volume-66579988',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9dfe6f88-03c6-4228-9a54-c9b55dff3a22,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8d4a57a44aef42c6ac17a7f245f7a90e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:34:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:34:38Z,use_quota=True,user_id='8a4211529de143a2876d1cf8fec15936',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:38.723390 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e007f1c7-5a8c-4176-945c-05ddaccdfc39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:38.729499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e007f1c7-5a8c-4176-945c-05ddaccdfc39) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:38.736784 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7dac6468-7e72-4a1e-b9e7-f71bd8f16f8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:38.742401 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7dac6468-7e72-4a1e-b9e7-f71bd8f16f8e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9dfe6f88-03c6-4228-9a54-c9b55dff3a22', 'volume_name': 'volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:38.749986 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (76605756-89db-4b98-ad0b-d103e571cae3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:38.753459 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (76605756-89db-4b98-ad0b-d103e571cae3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:38.765365 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a618237a-bd79-4c9c-b1ed-86cd9ffb7e31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:38.770699 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Volume 9dfe6f88-03c6-4228-9a54-c9b55dff3a22: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22', 'volume_size': 1} May 14 21:34:38.772022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] creating volume 'volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:34:38.774831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:39.446440 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-1a0fbeff-fb79-40a4-82a6-4b8636cb3aa8 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['c9c724cc-f5d9-4570-841a-2726f0a593a0', '045554a8-2c02-43f0-ba07-eda06d5348b5', '374a9c2a-9782-44fe-bf69-b3a983a765fb', 'e137ce8b-2ee8-4fd4-93f6-f31ae14a9ed5'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:39.779238 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1a0fbeff-fb79-40a4-82a6-4b8636cb3aa8 tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:34:39.786021 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-1a0fbeff-fb79-40a4-82a6-4b8636cb3aa8 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-011f772f-07ff-4ec2-be7f-0280aa275ad5-delete_snapshot'" released by "delete_snapshot" :: held 32.006s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:34:40.028347 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:34:40.028878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:34:40.032624 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a618237a-bd79-4c9c-b1ed-86cd9ffb7e31) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:40.037578 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6fd7f40b-a430-405a-b3f6-e9a36fc31455) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:34:40.241761 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Volume volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22 (9dfe6f88-03c6-4228-9a54-c9b55dff3a22): created successfully May 14 21:34:40.241761 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6fd7f40b-a430-405a-b3f6-e9a36fc31455) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:34:40.251700 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Flow 'volume_create_manager' (eba2500b-acb2-4994-93aa-568074bfe698) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:34:40.363370 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-64fa6d33-231a-4321-9d76-4dfe1f22da4d tempest-SnapshotsActionsTest-790580035 None] Created volume successfully. May 14 21:34:41.490970 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-acd86b49-07e4-4d57-908e-ffbda7bfb489 req-3904cb24-6527-453e-8ac1-e21a7d46ee6d tempest-VolumesSnapshotTestJSON-1533504236 None] Terminate volume connection completed successfully. May 14 21:34:41.492467 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-acd86b49-07e4-4d57-908e-ffbda7bfb489 req-3904cb24-6527-453e-8ac1-e21a7d46ee6d tempest-VolumesSnapshotTestJSON-1533504236 None] Deleting attachment 35e9eaa5-ae44-4a00-b9d8-896444bac6d6. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:34:41.527853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e6f2dc1-7e52-4344-b0d7-b7575c6252d8 tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:45.999754 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0e6f2dc1-7e52-4344-b0d7-b7575c6252d8 tempest-SnapshotsActionsTest-790580035 None] Create snapshot completed successfully May 14 21:34:48.185963 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-98ba7d0c-7945-45fa-8fdf-5a6cc38ae849 tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:48.780320 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-914110ad-3620-4952-adc3-beb5cc3bd0a5 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:52.400243 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-98ba7d0c-7945-45fa-8fdf-5a6cc38ae849 tempest-SnapshotsActionsTest-790580035 None] Create snapshot completed successfully May 14 21:34:52.407799 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-914110ad-3620-4952-adc3-beb5cc3bd0a5 tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:34:53.340934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-276bc7a5-3800-469b-8a0c-0a1ce9e1e64c tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-f18f9c72-5fc4-4412-ae78-6ca84b6d4bfe-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.028s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:34:53.342571 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-276bc7a5-3800-469b-8a0c-0a1ce9e1e64c tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:53.805734 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5b818f1d-2881-4537-b797-a0882039c36c tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-887d4d09-5b81-4f1a-8a26-2e641623e943-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:34:53.815275 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b818f1d-2881-4537-b797-a0882039c36c tempest-SnapshotsActionsTest-698218774 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:34:58.791810 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-276bc7a5-3800-469b-8a0c-0a1ce9e1e64c tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['ad14a8cc-a9b2-40a1-b13d-833ce9641183', 'a591e8f5-975a-472d-af21-52039d6f7c4a', 'bcbb31d1-ca3c-4fe1-8b0d-9b0a7a296efe', 'd6b5779e-ff0f-49ab-a56c-17c81e23d7af'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:58.797216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-5b818f1d-2881-4537-b797-a0882039c36c tempest-SnapshotsActionsTest-698218774 None] Created reservations ['c6914484-39d3-4d67-a739-2f5c23f5fc81', 'e0a55cd6-dafc-45dd-a95e-4e6c75a1a0d0', 'c701f6f8-3abd-4386-b7f3-68dbb86abf3e', '01692aa4-ce57-41dc-b9b3-4ca3fcdc7c79'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:34:59.130997 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-276bc7a5-3800-469b-8a0c-0a1ce9e1e64c tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:34:59.149100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-276bc7a5-3800-469b-8a0c-0a1ce9e1e64c tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-f18f9c72-5fc4-4412-ae78-6ca84b6d4bfe-delete_snapshot'" released by "delete_snapshot" :: held 5.808s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:34:59.246143 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5b818f1d-2881-4537-b797-a0882039c36c tempest-SnapshotsActionsTest-698218774 None] Delete snapshot completed successfully. May 14 21:34:59.255157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5b818f1d-2881-4537-b797-a0882039c36c tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-887d4d09-5b81-4f1a-8a26-2e641623e943-delete_snapshot'" released by "delete_snapshot" :: held 5.449s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:35:00.090671 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c6b37470-5783-4ce7-9b9d-5bd9083fc88a tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:00.496925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-11f7d650-86c5-496e-94bf-b305ce1ec0ef tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:03.662442 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:35:03.664882 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:03.791423 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:04.002134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:35:04.612094 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c6b37470-5783-4ce7-9b9d-5bd9083fc88a tempest-SnapshotsActionsTest-790580035 None] Create snapshot completed successfully May 14 21:35:05.356914 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-11f7d650-86c5-496e-94bf-b305ce1ec0ef tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:35:05.447978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9d9cbd07-d349-422a-9ab5-95dd83601a3e tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-538435d6-ccfb-4f8c-9d82-67704e32d18d-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:35:05.458184 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9d9cbd07-d349-422a-9ab5-95dd83601a3e tempest-SnapshotsActionsTest-698218774 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:05.843151 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-070ba21a-054f-4be0-9939-67819ebe595d req-6088c472-03a6-4a1a-a863-fcd2a8878386 tempest-ServerRescueNegativeTestJSON-879335362 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:35:06.803357 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-070ba21a-054f-4be0-9939-67819ebe595d req-6088c472-03a6-4a1a-a863-fcd2a8878386 tempest-ServerRescueNegativeTestJSON-879335362 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.959s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:35:06.804412 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-070ba21a-054f-4be0-9939-67819ebe595d req-6088c472-03a6-4a1a-a863-fcd2a8878386 tempest-ServerRescueNegativeTestJSON-879335362 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:35:06.922740 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-070ba21a-054f-4be0-9939-67819ebe595d req-6088c472-03a6-4a1a-a863-fcd2a8878386 tempest-ServerRescueNegativeTestJSON-879335362 None] Connection info returned from driver {'name': 'volumes/volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '9ee16695-14bc-482f-8ed9-e3a38eb3ecc6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '2c753ef0-92e3-4f99-939f-25e260a20acd'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:35:07.155796 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-070ba21a-054f-4be0-9939-67819ebe595d req-6088c472-03a6-4a1a-a863-fcd2a8878386 tempest-ServerRescueNegativeTestJSON-879335362 None] attachment_update completed successfully. May 14 21:35:10.128475 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-9d9cbd07-d349-422a-9ab5-95dd83601a3e tempest-SnapshotsActionsTest-698218774 None] Created reservations ['e65f424d-9ff0-44dd-a521-9924e0490574', 'f0dc8f82-2a1e-4932-a43f-9f6fbbab9260', '69fb9c15-73d9-48dd-9bdf-fd4b720653cd', '3f76eb09-0d69-431f-9a92-a0be95e032ff'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:35:10.443037 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9d9cbd07-d349-422a-9ab5-95dd83601a3e tempest-SnapshotsActionsTest-698218774 None] Delete snapshot completed successfully. May 14 21:35:10.467387 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9d9cbd07-d349-422a-9ab5-95dd83601a3e tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-538435d6-ccfb-4f8c-9d82-67704e32d18d-delete_snapshot'" released by "delete_snapshot" :: held 5.018s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:35:11.975684 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3d89dcaf-b282-4d9f-94c6-ae358a731bbb tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:16.410763 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3d89dcaf-b282-4d9f-94c6-ae358a731bbb tempest-SnapshotsActionsTest-790580035 None] Create snapshot completed successfully May 14 21:35:17.387986 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6e885145-58b5-4534-9b89-554ba58f5b8f tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-cc04025a-a0a8-4ec7-a9cf-45ad4f637c5b-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.018s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:35:17.406865 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6e885145-58b5-4534-9b89-554ba58f5b8f tempest-SnapshotsActionsTest-698218774 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:22.305092 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-6e885145-58b5-4534-9b89-554ba58f5b8f tempest-SnapshotsActionsTest-698218774 None] Created reservations ['5f43176a-70c6-4d04-bfe3-81de4e3b45c4', 'ff05d854-b2a4-4902-98cf-7361865a1475', '33ef165f-4c0b-4a27-ab2e-ae022a83df66', 'a2797e52-76ff-451e-b40b-e0dba5a0ff9b'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:35:22.660417 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6e885145-58b5-4534-9b89-554ba58f5b8f tempest-SnapshotsActionsTest-698218774 None] Delete snapshot completed successfully. May 14 21:35:22.680500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6e885145-58b5-4534-9b89-554ba58f5b8f tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-cc04025a-a0a8-4ec7-a9cf-45ad4f637c5b-delete_snapshot'" released by "delete_snapshot" :: held 5.287s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:35:23.539184 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ae51b884-08ea-4172-9f8d-8595615468f4 tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:27.817064 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-ff59fc35-332c-4664-b233-f36c9401a4a1 req-16752589-7479-4dd1-8b6d-b05f209fd6f0 tempest-VolumesSnapshotTestJSON-1533504236 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:35:28.718102 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-ff59fc35-332c-4664-b233-f36c9401a4a1 req-16752589-7479-4dd1-8b6d-b05f209fd6f0 tempest-VolumesSnapshotTestJSON-1533504236 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.901s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:35:28.722559 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ff59fc35-332c-4664-b233-f36c9401a4a1 req-16752589-7479-4dd1-8b6d-b05f209fd6f0 tempest-VolumesSnapshotTestJSON-1533504236 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c221be08-99e4-4f09-a858-6fffe81fd931', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'c221be08-99e4-4f09-a858-6fffe81fd931', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:35:28.846600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-ff59fc35-332c-4664-b233-f36c9401a4a1 req-16752589-7479-4dd1-8b6d-b05f209fd6f0 tempest-VolumesSnapshotTestJSON-1533504236 None] Connection info returned from driver {'name': 'volumes/volume-c221be08-99e4-4f09-a858-6fffe81fd931', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c221be08-99e4-4f09-a858-6fffe81fd931', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ad5aa9d8-7f54-4b7c-b2c0-75ab214e89cc'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:35:28.998277 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ae51b884-08ea-4172-9f8d-8595615468f4 tempest-SnapshotsActionsTest-790580035 None] Create snapshot completed successfully May 14 21:35:29.045214 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-ff59fc35-332c-4664-b233-f36c9401a4a1 req-16752589-7479-4dd1-8b6d-b05f209fd6f0 tempest-VolumesSnapshotTestJSON-1533504236 None] attachment_update completed successfully. May 14 21:35:30.313729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-61de9bb3-5364-451f-8344-6263d366d74b tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-4e7ecc4e-d818-4f30-8a31-71b90ae853fa-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.019s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:35:30.316587 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-61de9bb3-5364-451f-8344-6263d366d74b tempest-SnapshotsActionsTest-698218774 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:32.799289 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4e782a94-fc35-45b0-900f-76f5f945474f tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:35.810569 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-61de9bb3-5364-451f-8344-6263d366d74b tempest-SnapshotsActionsTest-698218774 None] Created reservations ['d306d507-95d3-45c1-a50f-572afb0ba7ea', 'd08cf854-bdfa-47b3-a060-c9ddf6ccc488', '3eef25d3-f065-4cd4-8b95-dc79a6c0c121', '68bfbe2c-f968-4384-bc42-4ac082f56093'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:35:36.074374 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-61de9bb3-5364-451f-8344-6263d366d74b tempest-SnapshotsActionsTest-698218774 None] Delete snapshot completed successfully. May 14 21:35:36.085980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-61de9bb3-5364-451f-8344-6263d366d74b tempest-SnapshotsActionsTest-698218774 None] Lock "b'cinder-4e7ecc4e-d818-4f30-8a31-71b90ae853fa-delete_snapshot'" released by "delete_snapshot" :: held 5.768s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:35:37.203987 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4e782a94-fc35-45b0-900f-76f5f945474f tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:35:37.690804 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bfc4347b-a37c-4baf-86c9-3a0eacaf3a3a tempest-SnapshotsActionsTest-790580035 None] Lock "b'cinder-f6eec589-1197-4d4f-96c5-0979edd05d99-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.026s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:35:37.702928 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bfc4347b-a37c-4baf-86c9-3a0eacaf3a3a tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:38.609267 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9ee63a81-d191-44f8-b61a-47ce42eb02e9 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:41.856082 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9ee63a81-d191-44f8-b61a-47ce42eb02e9 tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:35:42.538527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-bfc4347b-a37c-4baf-86c9-3a0eacaf3a3a tempest-SnapshotsActionsTest-790580035 None] Created reservations ['a80f8b9f-91e0-49f2-a95f-2d987efe98c5', 'c6082b89-d8b3-4e77-9edb-4cba33fd5645', '2e093cad-d8c9-4bc5-8671-7766d0c9d49d', '179cba91-44b6-43f3-85a8-98bb72a0be9d'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:35:42.825984 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e0fef791-62cf-4b12-a46f-f5aadc7c6188 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-d83313db-b8eb-46b2-91de-872cc39d75c3-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.043s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:35:42.829007 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e0fef791-62cf-4b12-a46f-f5aadc7c6188 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:43.219165 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bfc4347b-a37c-4baf-86c9-3a0eacaf3a3a tempest-SnapshotsActionsTest-790580035 None] Delete snapshot completed successfully. May 14 21:35:43.248910 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bfc4347b-a37c-4baf-86c9-3a0eacaf3a3a tempest-SnapshotsActionsTest-790580035 None] Lock "b'cinder-f6eec589-1197-4d4f-96c5-0979edd05d99-delete_snapshot'" released by "delete_snapshot" :: held 5.558s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:35:43.261640 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] Lock "b'cinder-9dfe6f88-03c6-4228-9a54-c9b55dff3a22-delete_volume'" acquired by "delete_volume" :: waited 0.047s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:35:43.464971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:43.742016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:35:43.743651 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] Volume volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:35:44.010648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] deleting rbd volume volume-9dfe6f88-03c6-4228-9a54-c9b55dff3a22 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:35:51.044822 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] Created reservations ['21846d2c-1330-4b39-80ab-322b3689223a', '42636117-6673-410d-9235-69303a944ad3', '358ca69a-e1be-4165-b7e0-03abaa2ae570', '9b2fb043-6ac5-4b37-a746-33a7382564ad'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:35:51.280236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:51.352750 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:35:51.448535 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:35:51.457021 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] Deleted volume successfully. May 14 21:35:51.532420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9479347b-2d00-41e3-bbfc-4416212d433f tempest-SnapshotsActionsTest-790580035 None] Lock "b'cinder-9dfe6f88-03c6-4228-9a54-c9b55dff3a22-delete_volume'" released by "delete_volume" :: held 8.270s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:02.116452 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d17b1d83-8a83-4a94-87b9-7688426f40d9 req-b18cbd60-3e5e-4576-a515-5e39da8b0ff3 tempest-TaggedBootDevicesTest-1853898337 None] Terminate volume connection completed successfully. May 14 21:36:02.117931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d17b1d83-8a83-4a94-87b9-7688426f40d9 req-b18cbd60-3e5e-4576-a515-5e39da8b0ff3 tempest-TaggedBootDevicesTest-1853898337 None] Deleting attachment f3ac3501-aff2-41ff-a7ba-a6b0e9645f89. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:36:02.379186 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d17b1d83-8a83-4a94-87b9-7688426f40d9 req-f85b511a-37d0-4254-a598-8059731d9a5e tempest-TaggedBootDevicesTest-1853898337 None] Terminate volume connection completed successfully. May 14 21:36:02.380314 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d17b1d83-8a83-4a94-87b9-7688426f40d9 req-f85b511a-37d0-4254-a598-8059731d9a5e tempest-TaggedBootDevicesTest-1853898337 None] Deleting attachment 5db66234-e93a-44df-84df-a38baa9ead09. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:36:02.642562 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d17b1d83-8a83-4a94-87b9-7688426f40d9 req-51dc9fc1-5b23-4931-9f9a-4682eadc4329 tempest-TaggedBootDevicesTest-1853898337 None] Terminate volume connection completed successfully. May 14 21:36:02.645691 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d17b1d83-8a83-4a94-87b9-7688426f40d9 req-51dc9fc1-5b23-4931-9f9a-4682eadc4329 tempest-TaggedBootDevicesTest-1853898337 None] Deleting attachment 747510e5-914e-4798-a034-87797f5044d5. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:36:03.665381 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:36:03.667923 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:03.741943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:03.828327 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:07.203971 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-efc37362-30e7-4d87-a04f-2021396c7fd4 req-67f31d18-3abb-478a-9c2b-6f4a261942f2 tempest-ServerRescueNegativeTestJSON-879335362 None] Terminate volume connection completed successfully. May 14 21:36:07.204568 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-efc37362-30e7-4d87-a04f-2021396c7fd4 req-67f31d18-3abb-478a-9c2b-6f4a261942f2 tempest-ServerRescueNegativeTestJSON-879335362 None] Deleting attachment 2c753ef0-92e3-4f99-939f-25e260a20acd. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:36:14.721142 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e0fef791-62cf-4b12-a46f-f5aadc7c6188 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['25279d65-eed6-454b-ad85-f7673fc61293', '4deac286-afee-4abb-a98f-68f735ed772f', '73b17851-6337-4b91-8cad-b90497fdf12c', '69b0965a-a64b-438c-884f-fe5bc5501153'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:15.080341 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e0fef791-62cf-4b12-a46f-f5aadc7c6188 tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:36:15.091212 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e0fef791-62cf-4b12-a46f-f5aadc7c6188 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-d83313db-b8eb-46b2-91de-872cc39d75c3-delete_snapshot'" released by "delete_snapshot" :: held 32.266s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:15.500227 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-7d2d29a8-c865-4570-a305-7473b65a3efb-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:15.696208 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b8482d4f-6f2d-4f8e-a964-20fc51091340 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-67f2abf6-9c78-45fa-a586-4297079164ca-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.030s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:15.702329 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b8482d4f-6f2d-4f8e-a964-20fc51091340 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:15.979704 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:16.240852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:36:16.241669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] Volume volume-7d2d29a8-c865-4570-a305-7473b65a3efb is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:36:16.532599 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] deleting rbd volume volume-7d2d29a8-c865-4570-a305-7473b65a3efb {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:36:19.680654 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] Lock "b'cinder-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6-delete_volume'" acquired by "delete_volume" :: waited 0.029s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:19.841200 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:20.200325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:36:20.204898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] Volume volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:36:20.496621 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] deleting rbd volume volume-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:36:21.606047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:36:21.639558 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Flow 'volume_create_manager' (a1c93cc1-68a9-4cbb-a830-854735f3fab2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:36:21.642191 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e348fad-cf1c-449d-9fd6-162c50204513) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:21.675188 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e348fad-cf1c-449d-9fd6-162c50204513) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:36:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesServicesTestJSON-Volume-1942716572',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5de2218a-0142-48d7-a6a7-3839c37d0fec,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6aadc80362174e27af7447356225b8a1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:36:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:36:21Z,use_quota=True,user_id='160298e5de5c4186a46f39c33df3b31e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:21.682282 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (37e395fe-6fbe-49bc-8b1f-e019cff8f939) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:21.698388 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (37e395fe-6fbe-49bc-8b1f-e019cff8f939) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:21.705873 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c2d4b9de-df48-441a-a134-92d091e37d27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:21.711052 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c2d4b9de-df48-441a-a134-92d091e37d27) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5de2218a-0142-48d7-a6a7-3839c37d0fec', 'volume_name': 'volume-5de2218a-0142-48d7-a6a7-3839c37d0fec', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:21.717034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (702ac85a-233b-4716-892a-dca6925f5195) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:21.724457 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (702ac85a-233b-4716-892a-dca6925f5195) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:21.729731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f9048c5f-fa93-486f-a806-e6bf01ee0920) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:21.731066 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Volume 5de2218a-0142-48d7-a6a7-3839c37d0fec: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5de2218a-0142-48d7-a6a7-3839c37d0fec', 'volume_size': 1} May 14 21:36:21.731657 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] creating volume 'volume-5de2218a-0142-48d7-a6a7-3839c37d0fec' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:36:21.732619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:22.658300 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] Created reservations ['8c4cc1e3-ccfd-4eb0-930d-ce477ea42fce', 'b44c6f13-c41f-4df1-a720-9a8bc849dd6c', 'b554d6d7-d1c0-428b-a574-6fdade0d1ee0', '6912f49a-9175-4cea-8268-343ca762af73'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:22.801195 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:36:22.801682 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:36:22.805785 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f9048c5f-fa93-486f-a806-e6bf01ee0920) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5de2218a-0142-48d7-a6a7-3839c37d0fec', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:22.810772 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (938e4575-01e3-4e00-88bc-be03aadb4376) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:22.941625 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Volume volume-5de2218a-0142-48d7-a6a7-3839c37d0fec (5de2218a-0142-48d7-a6a7-3839c37d0fec): created successfully May 14 21:36:22.947981 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (938e4575-01e3-4e00-88bc-be03aadb4376) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:22.970448 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Flow 'volume_create_manager' (a1c93cc1-68a9-4cbb-a830-854735f3fab2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:36:23.007435 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:23.102894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:23.179668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:23.182218 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a444130c-3183-4f49-ad50-abfcf0e82ef2 tempest-VolumesServicesTestJSON-1519174464 None] Created volume successfully. May 14 21:36:23.187171 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] Deleted volume successfully. May 14 21:36:23.308828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-17a68f04-9340-4255-bedb-e9922d441ec1 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-7d2d29a8-c865-4570-a305-7473b65a3efb-delete_volume'" released by "delete_volume" :: held 7.806s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:23.556420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-12a0e3c5-2efb-4eb6-a302-15e9e7d28509-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:23.684984 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:23.932568 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:36:23.933429 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] Volume volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:36:24.172961 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] deleting rbd volume volume-12a0e3c5-2efb-4eb6-a302-15e9e7d28509 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:36:24.198820 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] Lock "b'cinder-5de2218a-0142-48d7-a6a7-3839c37d0fec-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:24.321509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:24.600907 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:36:24.601655 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] Volume volume-5de2218a-0142-48d7-a6a7-3839c37d0fec is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:36:24.797011 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] deleting rbd volume volume-5de2218a-0142-48d7-a6a7-3839c37d0fec {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:36:26.896066 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] Created reservations ['a33feccd-dcee-48a2-8f7d-61ba5e269346', '06e4d33a-09c7-49ee-906d-f923a8dad5b6', '9be1a34d-994b-4aeb-9288-efbf20423ae6', '224fc5f5-cd4e-4d10-bed3-54f9b0e53666'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:27.247665 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:27.310921 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:27.418285 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:27.433468 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] Deleted volume successfully. May 14 21:36:27.505998 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4963a4d6-0d4d-4cc2-a36d-1bc95db537f0 tempest-ServerRescueNegativeTestJSON-879335362 None] Lock "b'cinder-9ee16695-14bc-482f-8ed9-e3a38eb3ecc6-delete_volume'" released by "delete_volume" :: held 7.826s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:27.699650 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] Lock "b'cinder-ba6689c8-5731-4271-ab38-0b3532441e18-delete_volume'" acquired by "delete_volume" :: waited 0.017s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:27.891460 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:28.301501 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:36:28.303022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] Volume volume-ba6689c8-5731-4271-ab38-0b3532441e18 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:36:28.526890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] deleting rbd volume volume-ba6689c8-5731-4271-ab38-0b3532441e18 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:36:33.057692 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] Created reservations ['121edfa9-a303-43cf-af9b-2e3b28f65f9b', 'b380214e-4762-4c6f-a01a-a81960b995ff', '2cdce129-2f49-4939-96c4-c7329a184e70', '75e3f8b9-986b-4c35-a32d-bf144cfa2ef0'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:33.310133 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:33.390223 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:33.454639 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:33.460046 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] Created reservations ['ddc13dd3-a9c7-49e7-8a86-9707f3a2a178', 'edc4b401-4197-4ee5-961b-fb498e006cad', '3749272b-f8de-4109-94f5-781675ea5b7e', '80b51b2b-c38c-4dc7-a555-08fd95346291'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:33.469769 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] Deleted volume successfully. May 14 21:36:33.602724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d3fc463d-05ac-4df0-8b71-b0e5725d5141 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-12a0e3c5-2efb-4eb6-a302-15e9e7d28509-delete_volume'" released by "delete_volume" :: held 10.046s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:33.752766 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:33.827901 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:33.937385 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:33.957398 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] Deleted volume successfully. May 14 21:36:33.985570 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-252f69af-c210-4b9b-8b32-e9dc857c2b60-delete_volume'" acquired by "delete_volume" :: waited 0.037s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:34.074857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3dea9054-e454-4026-91a9-a1137aec643a tempest-VolumesServicesTestJSON-1519174464 None] Lock "b'cinder-5de2218a-0142-48d7-a6a7-3839c37d0fec-delete_volume'" released by "delete_volume" :: held 9.876s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:34.142504 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:34.582831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:36:34.831440 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] deleting rbd volume volume-252f69af-c210-4b9b-8b32-e9dc857c2b60 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:36:35.121810 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] Created reservations ['b2a11088-d6ba-415d-815e-21fcd4769502', 'b910da3c-5b8c-4950-9636-c359951b17aa', '4099e316-c5ac-4b04-87bb-555d05a2a4dd', '3e10acd9-349e-4852-bb02-14942c6171e2'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:35.398499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:35.463189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:35.524414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:35.528028 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] Deleted volume successfully. May 14 21:36:35.622467 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-83270483-dde0-4bc4-aab0-6976db0bd85a tempest-ServerRescueNegativeTestJSON-879335362 None] Lock "b'cinder-ba6689c8-5731-4271-ab38-0b3532441e18-delete_volume'" released by "delete_volume" :: held 7.923s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:40.156300 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:36:40.330152 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] deleting parent snapshot volume-252f69af-c210-4b9b-8b32-e9dc857c2b60.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:36:42.817913 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] Created reservations ['9c2e8cab-ad35-4231-9a36-c18019832e60', '4611f1d3-b4e4-4f0b-bac1-4c13dc908358', 'b31cd80e-2ed1-499f-9b0a-451a37908e08', 'b78fcab9-8977-4607-b5a7-cd2e7142a9a4'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:43.107965 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:43.173713 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:43.344321 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:43.352664 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] Deleted volume successfully. May 14 21:36:43.471641 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0e317f95-ab69-4cc5-8ab8-632389f50f65 tempest-TaggedBootDevicesTest-1853898337 None] Lock "b'cinder-252f69af-c210-4b9b-8b32-e9dc857c2b60-delete_volume'" released by "delete_volume" :: held 9.486s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:47.109188 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-b8482d4f-6f2d-4f8e-a964-20fc51091340 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['8d88ab09-71c5-4921-9634-83cc0c0ad4eb', '7da3b42b-7c04-4ca4-89fd-625d264f3a9a', 'ec3658ca-2bdc-4e61-b633-73e111731ee3', '73a3340a-e89b-40d4-9cdd-26069a89c1aa'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:36:47.354593 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b8482d4f-6f2d-4f8e-a964-20fc51091340 tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:36:47.363313 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b8482d4f-6f2d-4f8e-a964-20fc51091340 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-67f2abf6-9c78-45fa-a586-4297079164ca-delete_snapshot'" released by "delete_snapshot" :: held 31.665s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:36:48.437258 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-2b2311b1-3c05-4f57-9420-b807cdf1f210 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-345d546b-8a00-439d-869c-7cdb873e83f9-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.013s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:36:48.441231 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b2311b1-3c05-4f57-9420-b807cdf1f210 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:53.684694 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:36:53.730738 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Flow 'volume_create_manager' (8bd89665-dcec-4219-98d4-1320aa5e8c39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:36:53.807590 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2f7317cc-10af-4933-ba26-543eeed3f806) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:53.826937 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2f7317cc-10af-4933-ba26-543eeed3f806) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:36:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesTest-volume-1543865223',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=92ef22fa-8b94-45ca-88cc-92ceb2690af9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a46379eb12e1415da18d84dd5e1cf7b8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:36:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:36:53Z,use_quota=True,user_id='33f8ff4ebd434eff9d18afb0ed75252f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(3b63ce2d-2680-4312-9c23-870dfd73b9ec),volume_type_id=3b63ce2d-2680-4312-9c23-870dfd73b9ec)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:53.833067 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b8c69cd-63fc-4e1f-87a0-e4eada28be4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:53.837868 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b8c69cd-63fc-4e1f-87a0-e4eada28be4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:53.843912 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (26bed3b2-d6a6-4cdf-8244-3c5a69e84d74) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:53.848942 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (26bed3b2-d6a6-4cdf-8244-3c5a69e84d74) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '92ef22fa-8b94-45ca-88cc-92ceb2690af9', 'volume_name': 'volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:53.854831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9f37550d-6ef0-466d-b9ff-713e1235c2cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:53.859156 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9f37550d-6ef0-466d-b9ff-713e1235c2cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:53.864571 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d15951e1-2a46-456a-989d-8c7cd46a506a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:53.865971 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Volume 92ef22fa-8b94-45ca-88cc-92ceb2690af9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9', 'volume_size': 1} May 14 21:36:53.866583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] creating volume 'volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:36:53.867610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:54.537243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:36:54.537919 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:36:54.542227 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d15951e1-2a46-456a-989d-8c7cd46a506a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:54.555468 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (10de94e5-b11c-4d2d-baea-cc884b191e86) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:36:54.668671 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Volume volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9 (92ef22fa-8b94-45ca-88cc-92ceb2690af9): created successfully May 14 21:36:54.672756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (10de94e5-b11c-4d2d-baea-cc884b191e86) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:36:54.679849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Flow 'volume_create_manager' (8bd89665-dcec-4219-98d4-1320aa5e8c39) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:36:54.777182 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-acad2f26-ffd4-49e0-a034-48363a0859fe tempest-VolumeTypesTest-1339013375 None] Created volume successfully. May 14 21:36:56.756978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5d98b9e-4051-4b86-a380-f7ee3e9c28bb tempest-VolumeTypesTest-1339013375 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:56.832814 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5d98b9e-4051-4b86-a380-f7ee3e9c28bb tempest-VolumeTypesTest-1339013375 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:36:56.906266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-b5d98b9e-4051-4b86-a380-f7ee3e9c28bb tempest-VolumeTypesTest-1339013375 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:36:56.911992 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b5d98b9e-4051-4b86-a380-f7ee3e9c28bb tempest-VolumeTypesTest-1339013375 None] Retype volume completed successfully. May 14 21:37:00.014730 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Lock "b'cinder-92ef22fa-8b94-45ca-88cc-92ceb2690af9-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:37:00.137813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:00.356600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:37:00.357248 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Volume volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:37:00.484035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] deleting rbd volume volume-92ef22fa-8b94-45ca-88cc-92ceb2690af9 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:37:03.662990 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:37:03.663623 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:03.845581 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:03.940247 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:37:04.990844 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-896369819 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-896369819, it is now deprecated. Please use the default quota class for default quota. May 14 21:37:04.992396 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-896369819 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-896369819, it is now deprecated. Please use the default quota class for default quota. May 14 21:37:05.129898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Created reservations ['7a65ca2b-e05d-4184-9d3c-e635a0386391', '3da6b3d0-19a7-42a6-8b16-bbcc1aa3a95b', '3a0a1d9b-e550-4ff7-95f9-ebd8ce8bb3ee', '9081ee7d-5588-43b0-b654-86283e559fb8'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:37:05.414600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:05.483574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:05.566309 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:37:05.570372 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Deleted volume successfully. May 14 21:37:05.658840 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e4c59e03-25a9-41f3-aaca-9791caed15b3 tempest-VolumeTypesTest-1339013375 None] Lock "b'cinder-92ef22fa-8b94-45ca-88cc-92ceb2690af9-delete_volume'" released by "delete_volume" :: held 5.644s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:37:19.462548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-2b2311b1-3c05-4f57-9420-b807cdf1f210 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['9b11d897-f487-4b62-86a6-8401e55009ec', '70c432a7-021a-4ef7-9156-219a86179da6', 'd199bc9a-3d49-4232-8ea0-81c378c992c0', '9c98f8f7-d3ea-4b79-8630-1ad208e0a490'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:37:19.676987 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2b2311b1-3c05-4f57-9420-b807cdf1f210 tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:37:19.683739 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-2b2311b1-3c05-4f57-9420-b807cdf1f210 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-345d546b-8a00-439d-869c-7cdb873e83f9-delete_snapshot'" released by "delete_snapshot" :: held 31.247s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:37:21.439456 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-4138fbef-040d-40e4-8988-0bf4ca963685 req-9caf0248-50ea-42e2-9db2-34ca5b49f908 tempest-VolumesSnapshotTestJSON-1533504236 None] Terminate volume connection completed successfully. May 14 21:37:21.439456 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-4138fbef-040d-40e4-8988-0bf4ca963685 req-9caf0248-50ea-42e2-9db2-34ca5b49f908 tempest-VolumesSnapshotTestJSON-1533504236 None] Deleting attachment ad5aa9d8-7f54-4b7c-b2c0-75ab214e89cc. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:37:31.059396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:37:31.090758 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (47f62684-f404-4609-9260-a8a75dc6c46a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:31.101801 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d28742cf-2218-428d-9b55-87f261d00c8b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:31.135695 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d28742cf-2218-428d-9b55-87f261d00c8b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:37:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-885368067',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='162610ac2697486d8a05f41713cc5ba2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:37:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:37:31Z,use_quota=True,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:31.142719 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb5f1008-3352-4e15-8d8d-3507769e3833) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:31.147440 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb5f1008-3352-4e15-8d8d-3507769e3833) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:31.153076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (253b5b63-721e-4d5c-8662-2977d54e139a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:31.157566 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (253b5b63-721e-4d5c-8662-2977d54e139a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3', 'volume_name': 'volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:31.163069 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b74c7b3f-8ab2-49d2-b282-00ba05736019) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:31.167258 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b74c7b3f-8ab2-49d2-b282-00ba05736019) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:31.172505 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ada2a082-96fa-4451-b844-098037e1c04b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:31.174361 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3', 'volume_size': 1} May 14 21:37:31.174961 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] creating volume 'volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:37:31.176248 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:32.010226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:37:32.010226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:37:32.010226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ada2a082-96fa-4451-b844-098037e1c04b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:32.016977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2c099e6d-c9c2-4c30-971c-ed8551a73a3a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:32.114305 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3 (ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3): created successfully May 14 21:37:32.117958 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2c099e6d-c9c2-4c30-971c-ed8551a73a3a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:32.123564 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (47f62684-f404-4609-9260-a8a75dc6c46a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:32.259155 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-11d8f8af-3f21-43aa-85f4-98cfb0cc2bc1 tempest-VolumesSnapshotTestJSON-1533504236 None] Created volume successfully. May 14 21:37:33.491807 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0220256e-6c01-4f82-96d7-cab4c6947984 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:36.249657 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0220256e-6c01-4f82-96d7-cab4c6947984 tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:37:37.947743 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:37:38.013091 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (567f283b-49d7-4fb7-9df9-0aed3165aaf5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:38.028808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8c0ec301-2c8d-4518-8e3b-af2a461a350d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:38.047976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8c0ec301-2c8d-4518-8e3b-af2a461a350d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:37:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-106142738',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6dee71be-9877-4d47-bc13-9e2d1484b418,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='162610ac2697486d8a05f41713cc5ba2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:37:38Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=73236b31-8a41-4875-94bd-47811467c62a,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:37:38Z,use_quota=True,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:38.053937 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5baeb614-4cf2-46f4-859d-85f0b0e822ec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:38.058364 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5baeb614-4cf2-46f4-859d-85f0b0e822ec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:38.064214 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (99ad5f65-1370-4c1f-985f-6cdd1046ca84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:38.068552 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (99ad5f65-1370-4c1f-985f-6cdd1046ca84) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '6dee71be-9877-4d47-bc13-9e2d1484b418', 'volume_name': 'volume-6dee71be-9877-4d47-bc13-9e2d1484b418', 'volume_size': 2, 'snapshot_id': '73236b31-8a41-4875-94bd-47811467c62a'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:38.074011 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6648c7fc-936c-4215-80b4-76d961c682ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:38.078029 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6648c7fc-936c-4215-80b4-76d961c682ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:38.084574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d7e1a07c-e56e-44cd-af05-ebc9c569bbbd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:38.085896 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume 6dee71be-9877-4d47-bc13-9e2d1484b418: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-6dee71be-9877-4d47-bc13-9e2d1484b418', 'volume_size': 2, 'snapshot_id': '73236b31-8a41-4875-94bd-47811467c62a'} May 14 21:37:38.107879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] cloning volumes/volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3@snapshot-73236b31-8a41-4875-94bd-47811467c62a to volume-6dee71be-9877-4d47-bc13-9e2d1484b418 {{(pid=123870) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}} May 14 21:37:38.108803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:38.175998 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:39.475196 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:37:39.475949 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:37:39.484314 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:40.726320 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d7e1a07c-e56e-44cd-af05-ebc9c569bbbd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6dee71be-9877-4d47-bc13-9e2d1484b418', 'volume_size': 2, 'snapshot_id': '73236b31-8a41-4875-94bd-47811467c62a'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:40.731402 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cbd78ccb-800f-4887-ac61-ca9ee825b836) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:40.831155 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-6dee71be-9877-4d47-bc13-9e2d1484b418 (6dee71be-9877-4d47-bc13-9e2d1484b418): created successfully May 14 21:37:40.836038 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cbd78ccb-800f-4887-ac61-ca9ee825b836) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:40.845834 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (567f283b-49d7-4fb7-9df9-0aed3165aaf5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:40.931577 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-de015345-2fed-4be1-b2bd-975410b4a671 tempest-VolumesSnapshotTestJSON-1533504236 None] Created volume successfully. May 14 21:37:41.361059 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-6dee71be-9877-4d47-bc13-9e2d1484b418-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:37:41.520901 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:41.884922 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:37:42.023977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-6dee71be-9877-4d47-bc13-9e2d1484b418 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:37:46.057514 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['7e556827-ce45-4b4b-97c1-eb48d6313e6e', 'a91e3766-f9c7-445a-a2fc-c6738832fceb', 'fd929234-b366-4132-8d93-507ad3640e75', 'd4a8011d-90ee-4e52-97e1-cc9d06c62f0e'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:37:46.331837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:46.403825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:46.452685 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:37:46.460056 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:37:46.592254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-82ac1262-87a2-48ed-b88e-e763fcbd3c32 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-6dee71be-9877-4d47-bc13-9e2d1484b418-delete_volume'" released by "delete_volume" :: held 5.231s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:37:47.869534 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:37:47.914949 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (762fabb7-3a44-4fbf-8f87-69ac09185f3b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:47.937244 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1d88ba84-b550-42fa-bd6b-1b5dfd35a921) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:47.985184 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1d88ba84-b550-42fa-bd6b-1b5dfd35a921) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:37:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1400971653',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8f698e5d-6710-457f-b901-a3cc224c65fb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='162610ac2697486d8a05f41713cc5ba2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:37:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:37:48Z,use_quota=True,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:47.993390 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b1f76afe-7887-4cd5-bc86-0900c978726f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:48.002535 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b1f76afe-7887-4cd5-bc86-0900c978726f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:48.010861 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (47662122-33fd-4316-b439-28577c6dca31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:48.027735 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (47662122-33fd-4316-b439-28577c6dca31) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8f698e5d-6710-457f-b901-a3cc224c65fb', 'volume_name': 'volume-8f698e5d-6710-457f-b901-a3cc224c65fb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:48.037708 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (df5e48d7-e677-4bd7-a7b6-56b181478a9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:48.043189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (df5e48d7-e677-4bd7-a7b6-56b181478a9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:48.048569 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (53a6d6df-67a7-4150-887c-d638ffba060c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:48.050138 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Volume 8f698e5d-6710-457f-b901-a3cc224c65fb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8f698e5d-6710-457f-b901-a3cc224c65fb', 'volume_size': 1} May 14 21:37:48.050583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] creating volume 'volume-8f698e5d-6710-457f-b901-a3cc224c65fb' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:37:48.051620 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:48.774932 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:37:48.775530 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:37:48.779638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (53a6d6df-67a7-4150-887c-d638ffba060c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8f698e5d-6710-457f-b901-a3cc224c65fb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:48.791171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f040cf1b-7bde-48c2-840e-8946a1f9ac64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:48.894593 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-8f698e5d-6710-457f-b901-a3cc224c65fb (8f698e5d-6710-457f-b901-a3cc224c65fb): created successfully May 14 21:37:48.899805 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f040cf1b-7bde-48c2-840e-8946a1f9ac64) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:48.930906 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (762fabb7-3a44-4fbf-8f87-69ac09185f3b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:49.030644 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5dbe6356-b016-46c8-a0e6-c6abead010cb tempest-VolumesSnapshotTestJSON-1533504236 None] Created volume successfully. May 14 21:37:50.752107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a6333ad2-1a3d-4879-a57e-89863b7333bb tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:53.525536 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a6333ad2-1a3d-4879-a57e-89863b7333bb tempest-VolumesSnapshotTestJSON-1533504236 None] Create snapshot completed successfully May 14 21:37:54.933043 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:37:55.000277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (6cf80209-04c9-48db-84fa-b5f5bbdc83f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:55.010877 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e93f2696-5c5f-4840-ab05-c4f129d2ccdf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:55.036630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e93f2696-5c5f-4840-ab05-c4f129d2ccdf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:37:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1805095619',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d14affde-06db-4898-929c-28ad13fa1e84,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='162610ac2697486d8a05f41713cc5ba2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:37:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=0fa46310-1e2b-48eb-813b-1cfdcc811641,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:37:55Z,use_quota=True,user_id='a8e967d05266477dbc5748ed77b8c69e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:55.044118 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (83ea121a-cd87-455a-a106-f5180f17cfe1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:55.056970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (83ea121a-cd87-455a-a106-f5180f17cfe1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:55.065336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (96255ee9-7dc3-463a-87bf-32c0e6865122) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:55.074047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (96255ee9-7dc3-463a-87bf-32c0e6865122) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'd14affde-06db-4898-929c-28ad13fa1e84', 'volume_name': 'volume-d14affde-06db-4898-929c-28ad13fa1e84', 'volume_size': 1, 'snapshot_id': '0fa46310-1e2b-48eb-813b-1cfdcc811641'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:55.085978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6d11d92a-6b12-4638-9d42-800dd2e01997) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:55.094239 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6d11d92a-6b12-4638-9d42-800dd2e01997) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:55.104477 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca9f8ae8-5585-42a2-8acb-dc72ac7c6126) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:55.108014 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Volume d14affde-06db-4898-929c-28ad13fa1e84: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-d14affde-06db-4898-929c-28ad13fa1e84', 'volume_size': 1, 'snapshot_id': '0fa46310-1e2b-48eb-813b-1cfdcc811641'} May 14 21:37:55.153935 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] cloning volumes/volume-8f698e5d-6710-457f-b901-a3cc224c65fb@snapshot-0fa46310-1e2b-48eb-813b-1cfdcc811641 to volume-d14affde-06db-4898-929c-28ad13fa1e84 {{(pid=123870) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}} May 14 21:37:55.155985 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:55.217166 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:56.464077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:37:56.465245 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:37:56.494733 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca9f8ae8-5585-42a2-8acb-dc72ac7c6126) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d14affde-06db-4898-929c-28ad13fa1e84', 'volume_size': 1, 'snapshot_id': '0fa46310-1e2b-48eb-813b-1cfdcc811641'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:56.500748 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b215c7d3-472d-4b17-be48-641c00611f18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:37:56.601795 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-d14affde-06db-4898-929c-28ad13fa1e84 (d14affde-06db-4898-929c-28ad13fa1e84): created successfully May 14 21:37:56.607173 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b215c7d3-472d-4b17-be48-641c00611f18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:37:56.624716 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Flow 'volume_create_manager' (6cf80209-04c9-48db-84fa-b5f5bbdc83f7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:37:56.750634 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c71c7fe7-de56-4862-980a-a1d7b8d6375d tempest-VolumesSnapshotTestJSON-1533504236 None] Created volume successfully. May 14 21:37:57.380813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-d14affde-06db-4898-929c-28ad13fa1e84-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:37:57.599851 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:37:57.864749 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:37:58.204360 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-d14affde-06db-4898-929c-28ad13fa1e84 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:38:02.510123 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['7d49c506-0b8c-4cfe-8d97-53c7b0aa13de', '5ffea5ae-c2be-4464-912d-eb176d49310c', 'c41d9932-040c-42ab-9ebe-3520bc8ede9a', '06ab7db7-1804-4d2b-b3e7-480154cbe1d3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:02.984787 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:03.091265 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:03.161469 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:38:03.175411 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:38:03.313768 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bd4454d1-aedf-4370-81e6-090eb6b121fa tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-d14affde-06db-4898-929c-28ad13fa1e84-delete_volume'" released by "delete_volume" :: held 5.933s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:03.368651 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e4f707af-73d5-442f-b895-145e3e350b26 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-0fa46310-1e2b-48eb-813b-1cfdcc811641-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.035s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:03.372599 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e4f707af-73d5-442f-b895-145e3e350b26 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:03.710558 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:38:03.712585 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:03.766517 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:03.832893 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:38:06.207240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e4f707af-73d5-442f-b895-145e3e350b26 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['bf37373b-83d0-4375-9a0d-bdef67660bd7', '21e0221f-3b6d-4f23-83b1-c52053fd886c', '7db9053f-7920-4f25-91f9-b4d0cd83f190', '5fcf18de-4d30-4027-92df-d68e362f2e86'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:06.492172 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e4f707af-73d5-442f-b895-145e3e350b26 tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:38:06.502631 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e4f707af-73d5-442f-b895-145e3e350b26 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-0fa46310-1e2b-48eb-813b-1cfdcc811641-delete_snapshot'" released by "delete_snapshot" :: held 3.134s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:06.794564 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-8f698e5d-6710-457f-b901-a3cc224c65fb-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:06.968422 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:07.190477 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:38:07.191288 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-8f698e5d-6710-457f-b901-a3cc224c65fb is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:38:07.279426 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-8f698e5d-6710-457f-b901-a3cc224c65fb {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:38:11.567683 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['b74224fb-63ac-4c60-9619-4a1c7566b23b', '9b33e3e2-78bc-4ea9-9fb1-8e6fde2cbb23', '0d23d7cc-f4a8-43b8-a634-2da5f31bc89d', '5b6efe69-2c92-4e03-bb56-8cf1e59663e8'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:11.862587 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:11.972860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:12.101586 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:38:12.105768 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:38:12.203294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-54b9c24d-ad38-44f0-9afb-6c1a4d9d6288 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-8f698e5d-6710-457f-b901-a3cc224c65fb-delete_volume'" released by "delete_volume" :: held 5.406s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:12.740800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4afdc2b4-ac7d-4f2c-bdae-b171b60c838a tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-73236b31-8a41-4875-94bd-47811467c62a-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.042s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:12.742380 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4afdc2b4-ac7d-4f2c-bdae-b171b60c838a tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:16.347547 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-4afdc2b4-ac7d-4f2c-bdae-b171b60c838a tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['45c592d1-493d-4262-9b4c-d19fb1c8cc3f', '92752b5d-126a-4eb6-8f17-5defbf5f8299', 'dddf98e1-3257-4151-93d0-000c71178916', '35baf518-7fbe-4a19-aaa7-6ef6fa5854a7'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:16.520371 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4afdc2b4-ac7d-4f2c-bdae-b171b60c838a tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:38:16.529146 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4afdc2b4-ac7d-4f2c-bdae-b171b60c838a tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-73236b31-8a41-4875-94bd-47811467c62a-delete_snapshot'" released by "delete_snapshot" :: held 3.787s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:17.297554 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3-delete_volume'" acquired by "delete_volume" :: waited 0.018s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:17.419439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:17.658220 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:38:17.658912 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:38:17.745608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:38:21.552304 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['f1b98d6d-3fa6-439b-aba2-4529b3bbfc15', '9b1daf55-1ac0-40ad-9083-86de9f0618b7', '77f12981-9d26-44f1-afcb-36f2180b5de8', '3c5523e0-d994-461f-83fb-d4b28cda2ead'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:21.830615 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:21.902671 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:21.977032 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:38:21.985528 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:38:22.090407 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d1a1e170-eaa0-415e-8c2a-a021b92be675 tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-ce04a1fc-f698-4b7b-abb5-b3f37eb5d5b3-delete_volume'" released by "delete_volume" :: held 4.793s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:23.240507 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-cd3708f6-e860-49e0-862f-98ce06d603c8-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:23.419467 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:23.645942 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:38:23.749808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-cd3708f6-e860-49e0-862f-98ce06d603c8 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:38:27.506433 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:38:27.661034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] deleting parent snapshot volume-cd3708f6-e860-49e0-862f-98ce06d603c8.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:38:29.990650 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['ae47aaa8-fcc6-4fbb-8bfa-24d07768f647', '34cc2447-9589-4504-810b-debf205054c4', 'd1592c1a-e322-4cd9-813b-1f31891d33f6', '03262d12-a003-4c20-9ca4-b28b5879440d'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:30.219439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:30.332659 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:30.437504 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:38:30.442133 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:38:30.536715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-22aded04-341b-4a8d-ab07-874d64fa14fd tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-cd3708f6-e860-49e0-862f-98ce06d603c8-delete_volume'" released by "delete_volume" :: held 7.296s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:31.230531 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5af87c3-e8dd-4160-addf-076cdb9d7d5f tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-f9966700-2a70-49c4-a5bc-d0c19d8baa1a-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:31.240834 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5af87c3-e8dd-4160-addf-076cdb9d7d5f tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:34.935285 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a5af87c3-e8dd-4160-addf-076cdb9d7d5f tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['f5196526-8aee-4283-878b-f6b1cc8eacdc', '057f45d6-6c6a-4fb2-a1db-b9c3b79991cd', '958753a7-f56b-4d86-b9b0-b38b25727480', '7927be34-db7f-4a5e-9b09-eb30a8046be1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:35.172890 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5af87c3-e8dd-4160-addf-076cdb9d7d5f tempest-VolumesSnapshotTestJSON-1533504236 None] Delete snapshot completed successfully. May 14 21:38:35.178108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5af87c3-e8dd-4160-addf-076cdb9d7d5f tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-f9966700-2a70-49c4-a5bc-d0c19d8baa1a-delete_snapshot'" released by "delete_snapshot" :: held 3.948s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:35.667528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-c221be08-99e4-4f09-a858-6fffe81fd931-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:38:35.745016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:35.921500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:38:35.921851 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] Volume volume-c221be08-99e4-4f09-a858-6fffe81fd931 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:38:36.007355 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] deleting rbd volume volume-c221be08-99e4-4f09-a858-6fffe81fd931 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:38:40.610811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] Created reservations ['77450ec1-31e1-4aab-b33c-1bec91c63b1c', '4cce0df1-1148-46be-a276-a1379d03158c', 'fe9a9134-0e54-442f-b1ec-22b532e54521', 'd22ae942-b2ed-4d44-9647-b8792a610489'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:38:40.845491 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:40.884482 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:40.929828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:38:40.932303 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] Deleted volume successfully. May 14 21:38:41.047771 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6a940db6-049a-408e-83ab-84cb3ed20f6b tempest-VolumesSnapshotTestJSON-1533504236 None] Lock "b'cinder-c221be08-99e4-4f09-a858-6fffe81fd931-delete_volume'" released by "delete_volume" :: held 5.381s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:38:46.486522 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:38:46.510731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Flow 'volume_create_manager' (78e4a0d4-5824-4b77-8502-133f4cf7b86f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:38:46.519915 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (333afa57-7366-4514-9871-cdea566e6c7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:38:46.540061 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (333afa57-7366-4514-9871-cdea566e6c7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:38:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotListTestJSON-Volume-226936598',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='74928faf25b04ff1ad55aeed69b6d573',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:38:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:38:46Z,use_quota=True,user_id='0fbfc456f35f40f7b068b0afbfb49633',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:38:46.549515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8b322e0b-21a5-4427-b4ec-cef17f1df380) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:38:46.554274 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8b322e0b-21a5-4427-b4ec-cef17f1df380) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:38:46.561317 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (85dcccd4-7095-4f37-b3d2-273fc6121e90) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:38:46.566866 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (85dcccd4-7095-4f37-b3d2-273fc6121e90) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a', 'volume_name': 'volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:38:46.574238 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4d9d57af-d304-4b22-b0d8-a235622f95da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:38:46.586640 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4d9d57af-d304-4b22-b0d8-a235622f95da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:38:46.590408 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84e1c84c-cff7-4132-9568-5ec52ddca48c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:38:46.591654 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Volume fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a', 'volume_size': 1} May 14 21:38:46.592234 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] creating volume 'volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:38:46.593192 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:47.175715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:38:47.176268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:38:47.183828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84e1c84c-cff7-4132-9568-5ec52ddca48c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:38:47.192185 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (126b7119-6b17-40c2-8079-ef0d09ae56e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:38:47.255641 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Volume volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a (fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a): created successfully May 14 21:38:47.262444 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (126b7119-6b17-40c2-8079-ef0d09ae56e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:38:47.275901 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Flow 'volume_create_manager' (78e4a0d4-5824-4b77-8502-133f4cf7b86f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:38:47.356431 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2bcb8d44-a16e-4672-b18c-4ca7a9798d3a tempest-VolumesSnapshotListTestJSON-1341809259 None] Created volume successfully. May 14 21:38:49.246557 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0328614-8525-4279-8ba1-3d9beb646fa5 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:51.707460 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b0328614-8525-4279-8ba1-3d9beb646fa5 tempest-VolumesSnapshotListTestJSON-1341809259 None] Create snapshot completed successfully May 14 21:38:52.812666 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5aa554bc-5628-4640-95b6-93c73941ce86 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:55.269351 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5aa554bc-5628-4640-95b6-93c73941ce86 tempest-VolumesSnapshotListTestJSON-1341809259 None] Create snapshot completed successfully May 14 21:38:56.548645 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e63caee6-e22f-423b-8cfc-cbc33ce1031e tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:38:59.358722 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e63caee6-e22f-423b-8cfc-cbc33ce1031e tempest-VolumesSnapshotListTestJSON-1341809259 None] Create snapshot completed successfully May 14 21:39:00.943310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-734f3611-b6d4-4daf-80e1-be847dee51a4 tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-d89131d8-d120-46ad-80ff-1b35696805fd-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.012s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:00.945909 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-734f3611-b6d4-4daf-80e1-be847dee51a4 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:01.491754 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:01.527989 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Flow 'volume_create_manager' (c9238375-36be-46a3-bd2a-dc64cde53d59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:01.536395 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (151b9d75-148b-4fd8-a2ca-61f4e789649e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:01.553546 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (151b9d75-148b-4fd8-a2ca-61f4e789649e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeSnapshotQuotasNegativeTestJSON-Volume-506357397',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=df46dae5-2174-425f-b825-a9bf54085bbf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3b15386d4c844f8095e31b707fc80d49',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:01Z,use_quota=True,user_id='0ca3fcd6c8d5419e98ac85d9238208d0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:01.560251 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (84277a30-4daa-46b6-8271-6dc6032181c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:01.564426 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (84277a30-4daa-46b6-8271-6dc6032181c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:01.569929 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5197b49e-eb91-4828-a8f0-5d45a66c2a05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:01.574266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5197b49e-eb91-4828-a8f0-5d45a66c2a05) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'df46dae5-2174-425f-b825-a9bf54085bbf', 'volume_name': 'volume-df46dae5-2174-425f-b825-a9bf54085bbf', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:01.580833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c0f891ab-c1ec-4dd1-b3d9-c5342880a294) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:01.585266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c0f891ab-c1ec-4dd1-b3d9-c5342880a294) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:01.590393 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3bb8da3f-5d56-4599-a080-5dcea71fa45d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:01.591693 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Volume df46dae5-2174-425f-b825-a9bf54085bbf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-df46dae5-2174-425f-b825-a9bf54085bbf', 'volume_size': 1} May 14 21:39:01.592096 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] creating volume 'volume-df46dae5-2174-425f-b825-a9bf54085bbf' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:01.592771 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:02.799676 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:02.799676 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:02.802948 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3bb8da3f-5d56-4599-a080-5dcea71fa45d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-df46dae5-2174-425f-b825-a9bf54085bbf', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:02.807366 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7ad9d09c-f74d-4a62-8fec-faafa029fd97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:02.893161 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Volume volume-df46dae5-2174-425f-b825-a9bf54085bbf (df46dae5-2174-425f-b825-a9bf54085bbf): created successfully May 14 21:39:02.897699 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7ad9d09c-f74d-4a62-8fec-faafa029fd97) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:02.903336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Flow 'volume_create_manager' (c9238375-36be-46a3-bd2a-dc64cde53d59) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:02.978173 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4d6af795-59f4-4d6d-8fe3-52c3c0ee0543 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Created volume successfully. May 14 21:39:03.716540 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:39:03.717406 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:03.798582 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:03.973310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:39:04.079043 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcd1d726-8932-44d0-8ccd-4e326273c593 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:05.443790 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-90705cc3-4ac8-44d6-8d1b-d6730c1354d6 tempest-BackendsCapabilitiesAdminTestsJSON-1497105908 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:5edbe80d-a532-477d-85bc-c68f9b0447a0:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=123870) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4733}} May 14 21:39:05.495765 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8849cdc0-a644-47d7-99ef-ba3652875e9e tempest-BackendsCapabilitiesAdminTestsJSON-1497105908 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:5edbe80d-a532-477d-85bc-c68f9b0447a0:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=123870) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4733}} May 14 21:39:05.595972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-734f3611-b6d4-4daf-80e1-be847dee51a4 tempest-VolumesSnapshotListTestJSON-1341809259 None] Created reservations ['c0fa1cd9-a0cd-4b54-a369-31795fe19876', 'b2436afb-ac1b-4c3d-83ce-03806185094b', 'bfadf41f-5e4a-4d88-ad53-bfcb4402c880', 'b677cabd-80c1-426f-b7a8-44de65df8d59'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:05.924354 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-734f3611-b6d4-4daf-80e1-be847dee51a4 tempest-VolumesSnapshotListTestJSON-1341809259 None] Delete snapshot completed successfully. May 14 21:39:05.929840 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-734f3611-b6d4-4daf-80e1-be847dee51a4 tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-d89131d8-d120-46ad-80ff-1b35696805fd-delete_snapshot'" released by "delete_snapshot" :: held 4.991s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:06.444914 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-1fe16260-7be2-420b-b86d-5b770402a8c1 tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-e1cab1ff-59d6-4f5a-aa14-b9dff3e78a09-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.019s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:06.446877 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1fe16260-7be2-420b-b86d-5b770402a8c1 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:07.391193 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bcd1d726-8932-44d0-8ccd-4e326273c593 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Create snapshot completed successfully May 14 21:39:08.805803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cb2fee11-e561-49fc-9376-7ac2e5b72d3d tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Lock "b'cinder-47987060-6507-479d-a42a-b6295d5b8165-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:08.808482 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb2fee11-e561-49fc-9376-7ac2e5b72d3d tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:10.533026 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-1fe16260-7be2-420b-b86d-5b770402a8c1 tempest-VolumesSnapshotListTestJSON-1341809259 None] Created reservations ['313ec7de-7001-4ebc-9d12-03ea162fdd1d', '5d320c49-1150-48d9-89d4-ba326f0188e1', '88f8793b-151f-42c3-9949-88763a4ba572', 'bcde3b23-25d9-44cc-bdc2-0562ddeac002'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:10.761687 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1fe16260-7be2-420b-b86d-5b770402a8c1 tempest-VolumesSnapshotListTestJSON-1341809259 None] Delete snapshot completed successfully. May 14 21:39:10.771384 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-1fe16260-7be2-420b-b86d-5b770402a8c1 tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-e1cab1ff-59d6-4f5a-aa14-b9dff3e78a09-delete_snapshot'" released by "delete_snapshot" :: held 4.326s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:11.805032 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-fc96b5e2-b765-4906-b90e-41c0e0a7498b tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-c55d8040-802b-4979-ac31-43cadac719dc-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:11.807667 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fc96b5e2-b765-4906-b90e-41c0e0a7498b tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:12.720258 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-cb2fee11-e561-49fc-9376-7ac2e5b72d3d tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Created reservations ['362aca7c-a8c0-4fff-88f6-e1f73d981884', '42a07ed6-b9e5-4dd8-8ee0-9563f3c26a4e', '8644425c-1e70-46c0-8867-01aba2cce12a', 'f0423e1d-3632-490f-aa4f-4e2597508aba'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:12.951419 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-cb2fee11-e561-49fc-9376-7ac2e5b72d3d tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Delete snapshot completed successfully. May 14 21:39:12.958016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cb2fee11-e561-49fc-9376-7ac2e5b72d3d tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Lock "b'cinder-47987060-6507-479d-a42a-b6295d5b8165-delete_snapshot'" released by "delete_snapshot" :: held 4.151s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:13.287365 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Lock "b'cinder-df46dae5-2174-425f-b825-a9bf54085bbf-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:13.414417 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:13.690023 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:39:13.690678 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Volume volume-df46dae5-2174-425f-b825-a9bf54085bbf is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:39:13.786905 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] deleting rbd volume volume-df46dae5-2174-425f-b825-a9bf54085bbf {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:39:15.844788 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-fc96b5e2-b765-4906-b90e-41c0e0a7498b tempest-VolumesSnapshotListTestJSON-1341809259 None] Created reservations ['887d3117-f3b5-45d2-b5c1-39abcb7f9d21', '7e98ddce-c978-4f49-899f-36cfb67547f1', '73222099-838f-48d1-bccc-dbb489c87b27', 'bfac3765-edaa-471c-9f75-e71b15e4ac5b'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:16.195427 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fc96b5e2-b765-4906-b90e-41c0e0a7498b tempest-VolumesSnapshotListTestJSON-1341809259 None] Delete snapshot completed successfully. May 14 21:39:16.204554 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-fc96b5e2-b765-4906-b90e-41c0e0a7498b tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-c55d8040-802b-4979-ac31-43cadac719dc-delete_snapshot'" released by "delete_snapshot" :: held 4.397s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:16.297953 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:16.407006 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:16.600308 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:39:16.600818 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] Volume volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:39:16.832949 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] deleting rbd volume volume-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:39:20.124367 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Created reservations ['9b96f7d4-b24a-4c3c-b43a-36ecc5fbee37', '42eddb16-101c-4e69-9f42-0d17dba8ba4e', 'c62a853a-8b6c-4272-8864-0769a15bd2d1', '80966c2c-79c3-4e5d-b4a1-ab1e83e1b6dd'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:20.400588 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:20.482047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:20.534850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:39:20.539323 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Deleted volume successfully. May 14 21:39:20.700693 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7487325a-af3d-4b7d-9a0e-1ff8b4f55583 tempest-VolumeSnapshotQuotasNegativeTestJSON-12461162 None] Lock "b'cinder-df46dae5-2174-425f-b825-a9bf54085bbf-delete_volume'" released by "delete_volume" :: held 7.409s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:22.410268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] Created reservations ['98f3571c-3078-4f23-8c28-546ce02b73e4', '42efb8af-d1ba-4e52-a086-8b3a50b22db3', '08771943-e2d0-4aa3-ba9f-713237f3b947', 'f6126e23-619b-4d0b-a673-53d7c9efe984'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:22.686694 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:22.726721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:22.787816 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:39:22.795061 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] Deleted volume successfully. May 14 21:39:22.927117 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5924cc1d-e30c-4644-9346-d4007cbaa7e9 tempest-VolumesSnapshotListTestJSON-1341809259 None] Lock "b'cinder-fdfcc910-6125-4e92-b9e5-4c2ca0e5f58a-delete_volume'" released by "delete_volume" :: held 6.629s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:28.752658 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-04fc754f-b6d0-4ad5-a04e-602c9f2e7c57 tempest-GroupsV320Test-1729157248 None] Group tempest-GroupsV320Test-Group-1605739902: creating May 14 21:39:28.863279 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-04fc754f-b6d0-4ad5-a04e-602c9f2e7c57 tempest-GroupsV320Test-1729157248 None] Group tempest-GroupsV320Test-Group-1605739902: created successfully May 14 21:39:28.879039 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-04fc754f-b6d0-4ad5-a04e-602c9f2e7c57 tempest-GroupsV320Test-1729157248 None] Create group completed successfully. May 14 21:39:29.802266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:29.822911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (0ba6ca40-4f67-42f2-9c5d-d8801bf2f3df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:29.829585 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (224b17e2-53b7-4b32-aaaf-e50e4944cbc8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:29.845454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (224b17e2-53b7-4b32-aaaf-e50e4944cbc8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-84286822',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=dba354d1-4c19-4d67-8ea4-7d3c352f7f7e,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:30Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:29.852803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e7db12cc-321e-4d93-b24a-1b36dc00490f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:29.859240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e7db12cc-321e-4d93-b24a-1b36dc00490f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:29.864513 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e0fdbd7f-1108-4890-93f7-18b13ada7a62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:29.868845 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e0fdbd7f-1108-4890-93f7-18b13ada7a62) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'dba354d1-4c19-4d67-8ea4-7d3c352f7f7e', 'volume_name': 'volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:29.876570 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e8f28c78-7ab7-4c1d-8f3b-5ad0aa19e872) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:29.881730 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e8f28c78-7ab7-4c1d-8f3b-5ad0aa19e872) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:29.886057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (58ce18b3-67d4-405c-8800-f9af367ea760) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:29.887322 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Volume dba354d1-4c19-4d67-8ea4-7d3c352f7f7e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e', 'volume_size': 1} May 14 21:39:29.887730 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] creating volume 'volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:29.888527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:30.514000 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:30.514664 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:30.518479 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (58ce18b3-67d4-405c-8800-f9af367ea760) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:30.523869 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7df6a29f-f60b-48b6-9beb-754cfcd7f03f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:30.615682 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Volume volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e (dba354d1-4c19-4d67-8ea4-7d3c352f7f7e): created successfully May 14 21:39:30.619602 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7df6a29f-f60b-48b6-9beb-754cfcd7f03f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:30.625878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (0ba6ca40-4f67-42f2-9c5d-d8801bf2f3df) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:30.685892 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-13443eee-ca6e-4b26-bbbb-4a6339dac89c tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:39:30.843266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a7fc2cdc-b7b5-432d-b5d7-80df54023e68 tempest-GroupsV320Test-1729157248 None] Created reservations ['e2ce3c74-e000-493b-9ae1-2c68b3aa06fe'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:31.064738 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a7fc2cdc-b7b5-432d-b5d7-80df54023e68 tempest-GroupsV320Test-1729157248 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:31.098555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a7fc2cdc-b7b5-432d-b5d7-80df54023e68 tempest-GroupsV320Test-1729157248 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:31.131077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-a7fc2cdc-b7b5-432d-b5d7-80df54023e68 tempest-GroupsV320Test-1729157248 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:39:31.137116 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a7fc2cdc-b7b5-432d-b5d7-80df54023e68 tempest-GroupsV320Test-1729157248 None] Delete group completed successfully. May 14 21:39:33.345057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:33.367373 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (25a996c7-446a-4662-b400-83a467b12930) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:33.374096 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69e133c7-5c67-4df6-af13-45e224268a7b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:33.392548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69e133c7-5c67-4df6-af13-45e224268a7b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:33Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1286940804',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6955e433-57fd-48e4-b5d1-2eb03e91f23a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:33Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:33.398107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7f8973f8-d1b9-48d0-b992-b29f813ccede) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:33.402384 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7f8973f8-d1b9-48d0-b992-b29f813ccede) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:33.407516 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3d8adca7-38b6-4a9a-a6f7-72bcecf6ae22) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:33.411931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3d8adca7-38b6-4a9a-a6f7-72bcecf6ae22) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6955e433-57fd-48e4-b5d1-2eb03e91f23a', 'volume_name': 'volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:33.416899 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a1748549-fbc4-4689-97aa-8c9c84a29a8f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:33.420970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a1748549-fbc4-4689-97aa-8c9c84a29a8f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:33.425613 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5009ee4a-671a-4fef-82e4-bbd16c8d38cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:33.426727 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Volume 6955e433-57fd-48e4-b5d1-2eb03e91f23a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a', 'volume_size': 1} May 14 21:39:33.427213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] creating volume 'volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:33.427844 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:34.233582 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:34.234207 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:34.242606 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5009ee4a-671a-4fef-82e4-bbd16c8d38cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:34.245014 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c6b02b2a-ad54-4100-801f-7c00a2277e11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:34.325353 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Volume volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a (6955e433-57fd-48e4-b5d1-2eb03e91f23a): created successfully May 14 21:39:34.332221 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c6b02b2a-ad54-4100-801f-7c00a2277e11) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:34.350292 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (25a996c7-446a-4662-b400-83a467b12930) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:34.438801 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-061160ab-5129-449b-a950-ad87dde8f54b tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:39:35.692076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-6955e433-57fd-48e4-b5d1-2eb03e91f23a-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:35.822753 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:36.047410 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:39:36.047410 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] Volume volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:39:36.151201 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-6955e433-57fd-48e4-b5d1-2eb03e91f23a {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:39:40.661883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] Created reservations ['7c5dbeb5-dc05-4870-b6f6-59778d5ba78a', 'b2afd2a5-8927-4aee-bf60-7c40b6faa898', '835ee524-5997-4d77-a2ac-c44ab57a0720', '59b73d01-0411-438b-b529-4ba9250c286e'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:41.000107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:41.168903 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:41.317567 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:39:41.342311 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:39:41.503255 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-f273d29d-df17-4114-bbbc-be866d677544 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-6955e433-57fd-48e4-b5d1-2eb03e91f23a-delete_volume'" released by "delete_volume" :: held 5.811s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:41.617793 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e-delete_volume'" acquired by "delete_volume" :: waited 0.028s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:41.832504 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:42.065675 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:39:42.065675 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] Volume volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:39:42.260419 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:39:46.360360 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] Created reservations ['56d1cfca-6655-49b8-9a38-7cee8e3cc1f4', '9b20bb66-7c5d-4975-85f1-da233e97178d', '6e0533f6-bb6e-4f02-a269-29e65455df52', '0089f4aa-1126-49b5-bf81-9b777e1440e9'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:39:46.612214 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:46.682077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:46.757006 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:39:46.767558 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:39:46.895047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-42f57f75-d943-4d0b-bbc2-e0f2a5f2a09d tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-dba354d1-4c19-4d67-8ea4-7d3c352f7f7e-delete_volume'" released by "delete_volume" :: held 5.277s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:39:48.384560 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:48.421670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (b2101710-b12b-4424-9420-02470a0cfc9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:48.428480 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (aa398886-608c-4109-a1d4-edcbe36c7e6d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:48.474441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (aa398886-608c-4109-a1d4-edcbe36c7e6d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-843276140',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cf7210ee-929c-4ef8-a22e-825b560ca0ff,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:48Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:48.495744 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c24bd6c1-934e-492a-9dbd-1dd60b7f4c23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:48.503729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c24bd6c1-934e-492a-9dbd-1dd60b7f4c23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:48.514575 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c797c159-0f5f-4292-ab0c-4691e0c81387) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:48.519336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c797c159-0f5f-4292-ab0c-4691e0c81387) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'volume_name': 'volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:48.528915 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dd560c96-49c5-4d92-96a3-5c2053e86581) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:48.529682 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dd560c96-49c5-4d92-96a3-5c2053e86581) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:48.534584 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (70a824e9-aa2f-486b-b9c3-05e09ca7977d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:48.535867 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Volume cf7210ee-929c-4ef8-a22e-825b560ca0ff: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'volume_size': 1} May 14 21:39:48.536529 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] creating volume 'volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:48.537710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:49.098908 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:49.099744 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:49.104317 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (70a824e9-aa2f-486b-b9c3-05e09ca7977d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:49.111901 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c465d7e4-2f6c-4a1a-b26d-a8d67396ab92) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:49.211300 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Volume volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff (cf7210ee-929c-4ef8-a22e-825b560ca0ff): created successfully May 14 21:39:49.215626 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c465d7e4-2f6c-4a1a-b26d-a8d67396ab92) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:49.223172 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (b2101710-b12b-4424-9420-02470a0cfc9d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:49.294997 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-188a9f97-6164-4b6a-a869-5a9c26f5c57b tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:39:51.280911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:51.316974 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Flow 'volume_create_manager' (96e83823-ffba-4159-a598-cc85285302dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:51.325619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e8f19fec-75b8-46d8-875a-4d7f68c42e38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.343589 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e8f19fec-75b8-46d8-875a-4d7f68c42e38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-1118980792',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=57fbd2b6-45fe-4e57-b40d-a67c55a0ff18,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='26300f67535247758fbda7e0000bec7a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:51Z,use_quota=True,user_id='1e691c70db224be383a5ba2458bdd400',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.350070 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2e4364a5-99fc-4387-a9ec-b396584a6446) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.354670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2e4364a5-99fc-4387-a9ec-b396584a6446) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.360790 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (be4393bd-3786-46a3-a7b1-655177de8051) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.377409 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (be4393bd-3786-46a3-a7b1-655177de8051) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '57fbd2b6-45fe-4e57-b40d-a67c55a0ff18', 'volume_name': 'volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.388413 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (812aead9-4463-417d-aeb0-c755dc8846b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.400111 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (812aead9-4463-417d-aeb0-c755dc8846b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.409057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (116e4c88-cd49-4e89-8dd4-2cc732d56b60) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.411420 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume 57fbd2b6-45fe-4e57-b40d-a67c55a0ff18: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18', 'volume_size': 1} May 14 21:39:51.411993 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] creating volume 'volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:51.412865 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:51.733368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:51.776134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Flow 'volume_create_manager' (6d00eeb8-5b23-449c-b3e4-172f198fc08d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:51.785111 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e150cb68-b98a-4f7a-af95-cba18085c008) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.803318 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e150cb68-b98a-4f7a-af95-cba18085c008) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotMetadataTestJSON-Volume-374553850',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f5e48706-cb67-452f-880c-b9daeaa4af25,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1c8d88a4468a4423a15fc177f693d93d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:51Z,use_quota=True,user_id='0f7b1f92d6fd4692b9c2ca865e0a01ab',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.812751 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e0c21eba-8953-4967-b67f-4eb82b86ecc5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.817413 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e0c21eba-8953-4967-b67f-4eb82b86ecc5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.823681 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (45bdb67c-49c4-4abe-9cc4-bb92e81f7d02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.830532 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (45bdb67c-49c4-4abe-9cc4-bb92e81f7d02) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f5e48706-cb67-452f-880c-b9daeaa4af25', 'volume_name': 'volume-f5e48706-cb67-452f-880c-b9daeaa4af25', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.836924 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (70b5f14b-a0b5-4634-abc5-3ee357000eed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.841725 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (70b5f14b-a0b5-4634-abc5-3ee357000eed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:51.846969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1123a69f-a3b4-4d72-ab87-aa5a66847ad7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:51.848245 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Volume f5e48706-cb67-452f-880c-b9daeaa4af25: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f5e48706-cb67-452f-880c-b9daeaa4af25', 'volume_size': 1} May 14 21:39:51.848704 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] creating volume 'volume-f5e48706-cb67-452f-880c-b9daeaa4af25' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:51.849515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:52.209151 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:52.214481 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:52.214605 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (116e4c88-cd49-4e89-8dd4-2cc732d56b60) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:52.219541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a6d8aa1c-fbbe-4f25-9f25-5e428cceae55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:52.246972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:52.277614 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (68b7e1e1-b4d0-472c-a7da-8c2c03b83a7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:52.284237 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (781635ae-4cdc-4e35-8a73-43f114df5284) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:52.301790 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (781635ae-4cdc-4e35-8a73-43f114df5284) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-663753510',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=09d39e1a-7e74-4657-9a27-0caa92401af9,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=cf7210ee-929c-4ef8-a22e-825b560ca0ff,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:52Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:52.307634 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ad7ccce-2e7a-4bb3-a918-bf9de80cff7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:52.312628 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ad7ccce-2e7a-4bb3-a918-bf9de80cff7f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:52.318090 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f8c54247-772f-4c1f-a851-187da3fe9e76) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:52.330169 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18 (57fbd2b6-45fe-4e57-b40d-a67c55a0ff18): created successfully May 14 21:39:52.333787 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a6d8aa1c-fbbe-4f25-9f25-5e428cceae55) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:52.340081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Flow 'volume_create_manager' (96e83823-ffba-4159-a598-cc85285302dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:52.350556 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f8c54247-772f-4c1f-a851-187da3fe9e76) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '09d39e1a-7e74-4657-9a27-0caa92401af9', 'volume_name': 'volume-09d39e1a-7e74-4657-9a27-0caa92401af9', 'volume_size': 1, 'source_volid': 'cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'source_volstatus': 'available'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:52.363595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5cbd69b4-0dce-45f2-91af-c19afbb12d28) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:52.363595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5cbd69b4-0dce-45f2-91af-c19afbb12d28) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:52.366486 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (484dd8d2-ee84-49ac-adad-e0ea80eb33c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:52.367659 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Volume 09d39e1a-7e74-4657-9a27-0caa92401af9: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-09d39e1a-7e74-4657-9a27-0caa92401af9', 'volume_size': 1, 'source_volid': 'cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'source_volstatus': 'available'} May 14 21:39:52.381206 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:52.779985 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] creating snapshot='volume-09d39e1a-7e74-4657-9a27-0caa92401af9.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:39:54.398180 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] cloning 'volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff@volume-09d39e1a-7e74-4657-9a27-0caa92401af9.clone_snap' to 'volume-09d39e1a-7e74-4657-9a27-0caa92401af9' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:39:54.407817 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1f0192f6-58d7-4392-9c9f-d9c74194439e tempest-VolumeQuotasAdminTestJSON-933456861 None] Created volume successfully. May 14 21:39:54.415065 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:54.415829 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:54.420169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1123a69f-a3b4-4d72-ab87-aa5a66847ad7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f5e48706-cb67-452f-880c-b9daeaa4af25', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:54.427097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d348ea61-dc84-401a-af42-405d3073a2c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:54.444116 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] Lock "b'cinder-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18-delete_volume'" acquired by "delete_volume" :: waited 0.038s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:39:54.515026 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Volume volume-f5e48706-cb67-452f-880c-b9daeaa4af25 (f5e48706-cb67-452f-880c-b9daeaa4af25): created successfully May 14 21:39:54.518709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d348ea61-dc84-401a-af42-405d3073a2c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:54.524649 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Flow 'volume_create_manager' (6d00eeb8-5b23-449c-b3e4-172f198fc08d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:54.609803 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-471c6d31-f2a3-4ddf-86ea-fe135ba35ef5 tempest-SnapshotMetadataTestJSON-1741793806 None] Created volume successfully. May 14 21:39:54.614104 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:54.976723 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:39:54.977165 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:39:55.257236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] deleting rbd volume volume-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:39:55.483955 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd7ec9a0-1a43-4fae-92b0-b06900937d88 tempest-SnapshotMetadataTestJSON-1741793806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:56.074968 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Volume volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:39:56.076282 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:56.076759 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:56.786287 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:39:56.789972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (484dd8d2-ee84-49ac-adad-e0ea80eb33c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-09d39e1a-7e74-4657-9a27-0caa92401af9', 'volume_size': 1, 'source_volid': 'cf7210ee-929c-4ef8-a22e-825b560ca0ff', 'source_volstatus': 'available'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:56.795730 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (330ce777-dcd3-4324-a4f8-ceeb55ae428d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:56.894845 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Volume volume-09d39e1a-7e74-4657-9a27-0caa92401af9 (09d39e1a-7e74-4657-9a27-0caa92401af9): created successfully May 14 21:39:56.898825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (330ce777-dcd3-4324-a4f8-ceeb55ae428d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:56.905175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (68b7e1e1-b4d0-472c-a7da-8c2c03b83a7a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:56.909654 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:56.933396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Flow 'volume_create_manager' (dc47912e-9f28-481e-8883-94d30553c481) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:56.940244 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cc854ca2-f08f-4488-90f5-231095fd879c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:56.960764 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cc854ca2-f08f-4488-90f5-231095fd879c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasNegativeTestJSON-Volume-1112434940',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a4519a9d445e45bdb08a0015c0b825b8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:56Z,use_quota=True,user_id='d53d1ce6bd334ed1ba8cc768134da62a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:56.968590 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f4108fe6-f8f3-4821-8474-01bad6f3a440) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:56.976157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f4108fe6-f8f3-4821-8474-01bad6f3a440) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:56.982978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3055b1b-7b72-45ea-97f0-184a3c5079d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:56.987368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3055b1b-7b72-45ea-97f0-184a3c5079d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8', 'volume_name': 'volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:56.992382 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d23ee501-696a-4e5e-a497-77b1fca0ae1e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:56.996504 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d23ee501-696a-4e5e-a497-77b1fca0ae1e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:57.001242 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0660e1d-1e4b-41b9-8fe0-589df9728b95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:57.002353 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Volume 1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8', 'volume_size': 1} May 14 21:39:57.002749 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] creating volume 'volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:57.003415 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:57.034348 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-57ca62c5-2cb2-40ba-8887-950234c37671 tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:39:58.114996 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:58.116161 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:58.121242 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0660e1d-1e4b-41b9-8fe0-589df9728b95) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:58.127456 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5cef89b9-d4b4-40af-aa60-eba1ac0d3a63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:58.217105 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Volume volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8 (1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8): created successfully May 14 21:39:58.225706 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5cef89b9-d4b4-40af-aa60-eba1ac0d3a63) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:58.232390 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Flow 'volume_create_manager' (dc47912e-9f28-481e-8883-94d30553c481) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:58.331423 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3b134647-d482-456e-adf6-d1e09d36edc4 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Created volume successfully. May 14 21:39:58.670406 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:39:58.694674 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (01ba82fa-0bb2-489c-bb55-1cf71df9323b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:39:58.702294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (67c21068-80d7-414f-9962-33dba244a10e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:58.720760 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (67c21068-80d7-414f-9962-33dba244a10e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:39:58Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-471827144',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6a3996a3-675e-4559-bcba-5592749128a5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:39:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:39:58Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:58.726817 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (89b681a5-abb8-4435-b0e8-19f1ac299401) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:58.738288 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (89b681a5-abb8-4435-b0e8-19f1ac299401) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:58.744572 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (792fdd9d-a3e7-436c-a6e6-ac91f8cade9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:58.750294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (792fdd9d-a3e7-436c-a6e6-ac91f8cade9e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6a3996a3-675e-4559-bcba-5592749128a5', 'volume_name': 'volume-6a3996a3-675e-4559-bcba-5592749128a5', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:58.758978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a02e9ff9-2500-499e-a285-ea88fd0c2367) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:58.768294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a02e9ff9-2500-499e-a285-ea88fd0c2367) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:58.781980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1e146bf-d09e-43b9-ad56-0407dccc2594) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:58.783308 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Volume 6a3996a3-675e-4559-bcba-5592749128a5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6a3996a3-675e-4559-bcba-5592749128a5', 'volume_size': 1} May 14 21:39:58.783883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] creating volume 'volume-6a3996a3-675e-4559-bcba-5592749128a5' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:39:58.788167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:39:59.808115 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bd7ec9a0-1a43-4fae-92b0-b06900937d88 tempest-SnapshotMetadataTestJSON-1741793806 None] Create snapshot completed successfully May 14 21:39:59.817071 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:39:59.817454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:39:59.827825 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1e146bf-d09e-43b9-ad56-0407dccc2594) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6a3996a3-675e-4559-bcba-5592749128a5', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:59.832643 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a5a25159-e134-4211-a3bc-0204d5730db2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:39:59.949217 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Volume volume-6a3996a3-675e-4559-bcba-5592749128a5 (6a3996a3-675e-4559-bcba-5592749128a5): created successfully May 14 21:39:59.962015 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a5a25159-e134-4211-a3bc-0204d5730db2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:39:59.969054 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (01ba82fa-0bb2-489c-bb55-1cf71df9323b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:00.028735 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1124fa34-cf5a-4848-824c-e44c8e4fbf4c tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:40:00.290798 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Lock "b'cinder-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8-delete_volume'" acquired by "delete_volume" :: waited 0.016s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:00.382777 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:00.631355 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:00.631813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Volume volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:00.849959 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] deleting rbd volume volume-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:00.871102 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-6a3996a3-675e-4559-bcba-5592749128a5-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:01.026942 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:01.370856 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:01.371292 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] Volume volume-6a3996a3-675e-4559-bcba-5592749128a5 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:01.634128 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-6a3996a3-675e-4559-bcba-5592749128a5 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:01.736660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] Created reservations ['1af45c8b-f1a2-45a2-8c4a-facaa0f45800', 'b8c4136f-61c0-4c1f-a9b7-35fb4025209d', '2c289cae-fb45-43ed-a18d-eaaa74157e87', '1b06fdbc-f0ad-48b5-aa96-c2f524d87fb5'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:01.923275 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:01.969668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:02.014589 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:02.019074 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] Deleted volume successfully. May 14 21:40:02.091312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8310b351-4df4-4284-bf08-58208150b462 tempest-SnapshotMetadataTestJSON-1741793806 None] Lock "b'cinder-6fcb74bb-8f88-4b7d-ac03-c9c3e72b1e3b-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:02.092860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8310b351-4df4-4284-bf08-58208150b462 tempest-SnapshotMetadataTestJSON-1741793806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:02.329338 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b5a245b1-dc39-43bb-9083-1e9e05eccfed tempest-VolumeQuotasAdminTestJSON-933456861 None] Lock "b'cinder-57fbd2b6-45fe-4e57-b40d-a67c55a0ff18-delete_volume'" released by "delete_volume" :: held 7.886s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:02.966859 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:02.986715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Flow 'volume_create_manager' (a09bce2b-6389-46c3-931d-a26d3311c7b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:02.994168 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96e7d670-1b5e-49b9-9617-4624cfe6cd29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:03.009839 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96e7d670-1b5e-49b9-9617-4624cfe6cd29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-223882389',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=985f6e03-ad77-4d4c-81eb-aa246a7e5cc8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='26300f67535247758fbda7e0000bec7a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:03Z,use_quota=True,user_id='1e691c70db224be383a5ba2458bdd400',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:03.016276 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7b231899-3a46-472a-99b2-1f8ba1a98452) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:03.020706 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7b231899-3a46-472a-99b2-1f8ba1a98452) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:03.026113 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c25efcb-3c1b-44ab-8e3d-5ee9f9b97916) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:03.030403 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c25efcb-3c1b-44ab-8e3d-5ee9f9b97916) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '985f6e03-ad77-4d4c-81eb-aa246a7e5cc8', 'volume_name': 'volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:03.035243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4f9cf43d-9a91-45c2-abb0-dbaefc19e24a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:03.039109 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4f9cf43d-9a91-45c2-abb0-dbaefc19e24a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:03.044879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0b71909a-31d8-4459-8a75-6394a08689da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:03.045957 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume 985f6e03-ad77-4d4c-81eb-aa246a7e5cc8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8', 'volume_size': 1} May 14 21:40:03.046346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] creating volume 'volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:03.047123 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:03.718293 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:40:03.719015 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:03.779713 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:03.807964 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:04.368521 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:04.369178 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:04.372917 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0b71909a-31d8-4459-8a75-6394a08689da) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:04.377656 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (371e7e33-0d22-4e8c-a6f3-5b8e67316256) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:04.460476 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Volume volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8 (985f6e03-ad77-4d4c-81eb-aa246a7e5cc8): created successfully May 14 21:40:04.464644 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (371e7e33-0d22-4e8c-a6f3-5b8e67316256) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:04.470333 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Flow 'volume_create_manager' (a09bce2b-6389-46c3-931d-a26d3311c7b3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:04.558095 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-86002034-54b1-49e2-adac-3f2b74ec9811 tempest-VolumeQuotasAdminTestJSON-933456861 None] Created volume successfully. May 14 21:40:06.341513 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-45e7e864-825c-46c3-9814-07c49d3f828b tempest-VolumeQuotasAdminTestJSON-528978289 None] Transfer volume completed successfully. May 14 21:40:07.216191 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-8310b351-4df4-4284-bf08-58208150b462 tempest-SnapshotMetadataTestJSON-1741793806 None] Created reservations ['04130a39-4c49-43cb-8911-3519bbd129df', 'd60f5dc1-c570-4959-82fa-cba300d98fa6', '9e23c272-78a6-45a1-9ddd-e16aa4c9bb96', '40f4657b-3a59-42a6-9a34-0c88f197eea0'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:07.418807 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] Lock "b'cinder-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8-delete_volume'" acquired by "delete_volume" :: waited 0.020s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:07.549528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:07.844704 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:07.845134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] Volume volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:08.108661 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] deleting rbd volume volume-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:08.111635 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8310b351-4df4-4284-bf08-58208150b462 tempest-SnapshotMetadataTestJSON-1741793806 None] Delete snapshot completed successfully. May 14 21:40:08.116964 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8310b351-4df4-4284-bf08-58208150b462 tempest-SnapshotMetadataTestJSON-1741793806 None] Lock "b'cinder-6fcb74bb-8f88-4b7d-ac03-c9c3e72b1e3b-delete_snapshot'" released by "delete_snapshot" :: held 6.026s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:08.539992 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] Lock "b'cinder-f5e48706-cb67-452f-880c-b9daeaa4af25-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:08.544882 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Created reservations ['aa39dc87-6e4f-41ab-a5ca-0aa3f291f1f5', '186f2abb-1cb2-47f5-8060-749c6a047f75', '09172e13-f4c2-41a9-9ab5-69514520623a', 'c9ddc99e-ece0-475f-9dbb-ee42ddf09a29'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:08.727719 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:09.033954 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:09.034734 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] Volume volume-f5e48706-cb67-452f-880c-b9daeaa4af25 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:09.388527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] deleting rbd volume volume-f5e48706-cb67-452f-880c-b9daeaa4af25 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:09.538644 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:09.579441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:09.634034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:09.641077 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Deleted volume successfully. May 14 21:40:09.839608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] Created reservations ['ab3fcda7-63ac-423a-82db-b05b87ad9922', '2338a4a9-4f0d-4fff-910f-59d86cc3f4e6', '5531878a-c4b6-4e1f-8411-ae7532c38184', 'c2e98044-a0e7-4ea4-aa26-0f666e741004'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:09.861763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-1be530ce-f17e-4f57-9912-2d36a839b018 tempest-VolumeQuotasNegativeTestJSON-191466567 None] Lock "b'cinder-1ab2d5ea-6495-4112-a3f5-f95b7cfcf7c8-delete_volume'" released by "delete_volume" :: held 9.572s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:10.127450 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:10.280793 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:10.324111 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:10.328851 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:40:10.521191 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-66bfe2ed-e06b-44cb-bd90-a2c190d8f2fb tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-6a3996a3-675e-4559-bcba-5592749128a5-delete_volume'" released by "delete_volume" :: held 9.650s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:10.576884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-09d39e1a-7e74-4657-9a27-0caa92401af9-delete_volume'" acquired by "delete_volume" :: waited 0.028s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:10.751134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:11.015201 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:11.362894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-09d39e1a-7e74-4657-9a27-0caa92401af9 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:16.188063 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] Created reservations ['307a385b-acc4-44de-af8f-7084a617f702', 'b72eb408-1308-470c-b6c6-f82baec075d9', '2729335f-1a19-4f89-815c-b83ea82c5264', '15c40ecf-4401-436a-876b-4d7346098c0c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:16.423455 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:16.470477 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:16.503602 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:16.507995 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] Deleted volume successfully. May 14 21:40:16.603108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7af06682-d6c5-479a-9d70-7a07e50c39d7 tempest-VolumeQuotasAdminTestJSON-2012132475 None] Lock "b'cinder-985f6e03-ad77-4d4c-81eb-aa246a7e5cc8-delete_volume'" released by "delete_volume" :: held 9.184s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:17.043603 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] Created reservations ['1ca3ce84-a20e-4d77-af54-14b53bc373c5', '7a7e88b7-9af2-4610-ad84-bd2a9e63bc48', '10db5d31-9c18-4ab7-9d77-b75684c89960', 'cc7e3d06-8a71-45e9-932d-4d1c39d62395'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:17.336665 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:17.363157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:17.392151 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:17.396966 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] Deleted volume successfully. May 14 21:40:17.550739 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-27538c56-c953-4ea9-835f-9b79c6b48a01 tempest-SnapshotMetadataTestJSON-1741793806 None] Lock "b'cinder-f5e48706-cb67-452f-880c-b9daeaa4af25-delete_volume'" released by "delete_volume" :: held 9.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:18.371511 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:40:18.563737 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] Volume volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:18.564175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] deleting parent snapshot volume-09d39e1a-7e74-4657-9a27-0caa92401af9.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:40:21.176618 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] Created reservations ['92081e8e-b050-48aa-803c-ce7f60f4a1ae', '0c3ba417-6b64-4f48-9120-8ad1cc23e3fd', 'bb7dd8ae-f3c2-4950-9c33-c0ec78083794', 'aed87176-3409-4f0a-b6da-f0e94339f24c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:21.362383 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:21.441235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:21.486165 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:21.499862 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:40:21.615608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-92b588d4-885f-4a28-b677-b1fa2f46eff1 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-09d39e1a-7e74-4657-9a27-0caa92401af9-delete_volume'" released by "delete_volume" :: held 11.038s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:22.520111 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:22.563604 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (cb2609ec-1e24-4b45-a054-e82695bca37c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:22.573884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a4fdcac1-2271-40c1-90bf-7eb9ecbc2997) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:22.595403 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a4fdcac1-2271-40c1-90bf-7eb9ecbc2997) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-363194536',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:22Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:22.601565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4313c211-abeb-4e2d-a415-3a420be74dea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:22.606717 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4313c211-abeb-4e2d-a415-3a420be74dea) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:22.612167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (df7a8cee-c255-4a49-be97-4ef5946403df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:22.738736 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (df7a8cee-c255-4a49-be97-4ef5946403df) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50', 'volume_name': 'volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:22.745020 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (20873fdf-b322-43e9-83c9-64353dc67b8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:22.748425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (20873fdf-b322-43e9-83c9-64353dc67b8e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:22.754463 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a642d0f2-948c-4b28-b357-880591efb087) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:22.758808 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Volume 45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:40:22.758808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Cloning 45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:40:22.768076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:22.877280 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 40, 22, 772177)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:40:22.878057 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:40:22.878225 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:40:22.892346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.118s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:22.892346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:40:22.892346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 40, 22, 890106)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:40:22.892881 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:40:22.893269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:40:22.904736 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:23.139699 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] creating snapshot='volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:40:24.439578 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50.clone_snap' to 'volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:40:24.559116 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:24.586280 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Flow 'volume_create_manager' (f86b494e-328e-4be1-88b5-6c79e6bce9d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:24.610466 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e6409e7-ff2f-4a44-9634-be4d9001e93f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:24.629469 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e6409e7-ff2f-4a44-9634-be4d9001e93f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Volume-848870136',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=af0dc711-750a-44a0-b81b-0754b812e053,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='537c4ac112c846fcbbd4aa124542ab64',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:23Z,use_quota=True,user_id='69c5e3ab4e494335839ffd7afb2b9c52',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:24.634939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (06364e4c-368f-4539-8373-04ee994549c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:24.639315 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (06364e4c-368f-4539-8373-04ee994549c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:24.644555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (feca00b3-c688-43f3-bfe7-c7b3f5219eb6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:24.649297 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (feca00b3-c688-43f3-bfe7-c7b3f5219eb6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'af0dc711-750a-44a0-b81b-0754b812e053', 'volume_name': 'volume-af0dc711-750a-44a0-b81b-0754b812e053', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:24.654704 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (20d2d003-fbe6-4ec5-80f9-6fe594418a88) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:24.659090 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (20d2d003-fbe6-4ec5-80f9-6fe594418a88) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:24.664499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (10a9d18c-d4c9-43f6-ba53-f65ef28d0f9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:24.666501 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Volume af0dc711-750a-44a0-b81b-0754b812e053: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-af0dc711-750a-44a0-b81b-0754b812e053', 'volume_size': 1} May 14 21:40:24.667076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] creating volume 'volume-af0dc711-750a-44a0-b81b-0754b812e053' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:24.668171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:25.419383 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:25.442813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Flow 'volume_create_manager' (95f3a7d8-c408-4ba9-b07a-a8b3d92811c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:25.452597 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4be1880f-39d5-4704-b965-962f90c92c61) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:25.474306 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4be1880f-39d5-4704-b965-962f90c92c61) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-2044663724',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=74bb8a05-2411-40ba-acdf-f6a7442d9843,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d8db1a4b246e4ae1aa99962f35e65510',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:25Z,use_quota=True,user_id='0397fa67544d46d6acbbe6d59a21b5eb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:25.481386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (809324b0-65c5-47c1-a76c-1bd0948c0efa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:25.485697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (809324b0-65c5-47c1-a76c-1bd0948c0efa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:25.490960 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db1efa65-2fdf-4447-9d88-0504be21ab59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:25.495688 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db1efa65-2fdf-4447-9d88-0504be21ab59) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '74bb8a05-2411-40ba-acdf-f6a7442d9843', 'volume_name': 'volume-74bb8a05-2411-40ba-acdf-f6a7442d9843', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:25.505451 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a4dc91c2-7ec7-43bb-bb66-cc2656e5483e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:25.509917 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a4dc91c2-7ec7-43bb-bb66-cc2656e5483e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:25.514995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7dc179d1-3443-4e1f-84ae-222715778d8d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:25.516131 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Volume 74bb8a05-2411-40ba-acdf-f6a7442d9843: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-74bb8a05-2411-40ba-acdf-f6a7442d9843', 'volume_size': 1} May 14 21:40:25.516678 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] creating volume 'volume-74bb8a05-2411-40ba-acdf-f6a7442d9843' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:25.517489 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:25.662077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:25.662890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:25.669304 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (10a9d18c-d4c9-43f6-ba53-f65ef28d0f9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-af0dc711-750a-44a0-b81b-0754b812e053', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:25.675934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f1604ddf-c3df-4d1d-853a-34cf113ababf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:25.783419 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Volume volume-af0dc711-750a-44a0-b81b-0754b812e053 (af0dc711-750a-44a0-b81b-0754b812e053): created successfully May 14 21:40:25.791972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f1604ddf-c3df-4d1d-853a-34cf113ababf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:25.805717 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Flow 'volume_create_manager' (f86b494e-328e-4be1-88b5-6c79e6bce9d5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:25.908064 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-88972890-28eb-48f4-a676-678db0def55d tempest-VolumesBackupsV39Test-500200030 None] Created volume successfully. May 14 21:40:26.171248 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:26.172565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:26.172939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:26.714043 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:40:26.714737 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Marking volume 45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:40:26.722655 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:26.723031 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:26.726744 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7dc179d1-3443-4e1f-84ae-222715778d8d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-74bb8a05-2411-40ba-acdf-f6a7442d9843', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:26.732642 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (00400998-bf2f-45c8-8a0e-39e917de23e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:26.762991 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:40:26Z,data_timestamp=2022-05-14T21:40:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-1007209795',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=68940f14-10a2-4082-adee-123b6a3059f8,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='537c4ac112c846fcbbd4aa124542ab64',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='69c5e3ab4e494335839ffd7afb2b9c52',volume_id=af0dc711-750a-44a0-b81b-0754b812e053) May 14 21:40:26.763763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] continue_backup in rpcapi backup_id 68940f14-10a2-4082-adee-123b6a3059f8 {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:40:26.816783 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Volume volume-74bb8a05-2411-40ba-acdf-f6a7442d9843 (74bb8a05-2411-40ba-acdf-f6a7442d9843): created successfully May 14 21:40:26.827011 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (00400998-bf2f-45c8-8a0e-39e917de23e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:26.836499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Flow 'volume_create_manager' (95f3a7d8-c408-4ba9-b07a-a8b3d92811c4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:26.840574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:40:26.841002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Creating volume glance metadata for volume 45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:40:26.928084 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:40:26.952866 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c83d9932-b4d0-43ce-9ef2-36c2223bc752 tempest-AbsoluteLimitsTests-1102377737 None] Created volume successfully. May 14 21:40:26.999999 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a642d0f2-948c-4b28-b357-880591efb087) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:27.006587 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ffeceb76-8221-4c02-8af4-b8ee712b9029) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:27.113264 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Volume volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50 (45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50): created successfully May 14 21:40:27.117522 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ffeceb76-8221-4c02-8af4-b8ee712b9029) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:27.125858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (cb2609ec-1e24-4b45-a054-e82695bca37c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:27.210740 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b9ead562-6133-4fe1-b82d-436b4de17e48 tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:40:27.451832 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:27.473741 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Flow 'volume_create_manager' (99baa63c-41f2-4914-90b8-f400e5908cc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:27.480745 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e2833fc-ef3c-4c33-bee5-aa8564038858) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:27.502818 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e2833fc-ef3c-4c33-bee5-aa8564038858) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeRetypeWithoutMigrationTest-Volume-1154272873',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=28ef052f-001b-41b0-80a6-22122e8dd2d0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='12839e8098d74d729bceddeda4fa1acb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:27Z,use_quota=True,user_id='3bfc2374351c422ca49f5ef479322e3e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(7c811f8e-5fdc-48c1-a841-b40b97cb2f43),volume_type_id=7c811f8e-5fdc-48c1-a841-b40b97cb2f43)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:27.508583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fa7f300-4d14-4143-ad54-5be1b65e042a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:27.514182 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fa7f300-4d14-4143-ad54-5be1b65e042a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:27.519836 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cfcdd9e0-ce26-4203-993c-6897e88d1187) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:27.525413 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cfcdd9e0-ce26-4203-993c-6897e88d1187) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '28ef052f-001b-41b0-80a6-22122e8dd2d0', 'volume_name': 'volume-28ef052f-001b-41b0-80a6-22122e8dd2d0', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:27.530609 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3be80d45-3839-4040-aefc-bccb00983939) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:27.534793 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3be80d45-3839-4040-aefc-bccb00983939) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:27.539879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (91a6d227-0ca7-42c3-b214-1ee715a83b03) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:27.542899 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Volume 28ef052f-001b-41b0-80a6-22122e8dd2d0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-28ef052f-001b-41b0-80a6-22122e8dd2d0', 'volume_size': 1} May 14 21:40:27.543660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] creating volume 'volume-28ef052f-001b-41b0-80a6-22122e8dd2d0' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:27.545210 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:27.697312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] Lock "b'cinder-74bb8a05-2411-40ba-acdf-f6a7442d9843-delete_volume'" acquired by "delete_volume" :: waited 0.024s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:27.705199 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.777s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:40:27.706499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-af0dc711-750a-44a0-b81b-0754b812e053', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'af0dc711-750a-44a0-b81b-0754b812e053', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:40:27.732327 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] Initialize volume connection completed successfully. May 14 21:40:27.829629 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:28.073972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:28.076370 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] Volume volume-74bb8a05-2411-40ba-acdf-f6a7442d9843 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:28.393934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] deleting rbd volume volume-74bb8a05-2411-40ba-acdf-f6a7442d9843 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:28.524898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:28.526312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:28.531648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (91a6d227-0ca7-42c3-b214-1ee715a83b03) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-28ef052f-001b-41b0-80a6-22122e8dd2d0', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:28.541836 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b6128fae-ae4b-4ae0-bd89-f62fe606d223) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:28.638471 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Volume volume-28ef052f-001b-41b0-80a6-22122e8dd2d0 (28ef052f-001b-41b0-80a6-22122e8dd2d0): created successfully May 14 21:40:28.642211 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b6128fae-ae4b-4ae0-bd89-f62fe606d223) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:28.648301 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Flow 'volume_create_manager' (99baa63c-41f2-4914-90b8-f400e5908cc6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:28.722886 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-14652009-9d52-413a-8210-84bd045f885a tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Created volume successfully. May 14 21:40:29.107433 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:29.128248 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (1cd65996-e86f-4464-9359-1a37938a3a50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:29.135294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ffba4732-599d-405b-8263-b36c7b20181b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.154977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ffba4732-599d-405b-8263-b36c7b20181b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:28Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-409606635',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c0542095f84f98a91c9df1925426f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:29Z,use_quota=True,user_id='e8c801f66f3e403cac2618dfeb7be51c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.160521 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4e2ac9d2-2d4f-4928-a859-c3d6d685baff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.164828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4e2ac9d2-2d4f-4928-a859-c3d6d685baff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.170602 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (10630e1e-df3d-46b3-a0e8-6d24a9390e5f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.174758 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (10630e1e-df3d-46b3-a0e8-6d24a9390e5f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b', 'volume_name': 'volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.179811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b4f4938c-a4a9-4f07-9287-465b788d6b5d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.185254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b4f4938c-a4a9-4f07-9287-465b788d6b5d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.203939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (742c254a-64c7-4007-a3d7-99aac53856bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.203939 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Volume 7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b', 'volume_size': 1} May 14 21:40:29.203939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] creating volume 'volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:29.203939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:29.333871 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:29.352488 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Flow 'volume_create_manager' (21090453-3e5a-4956-8304-d73c9bd4bfdc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:29.359515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ff105905-d7be-4106-ba66-bb69a8db5313) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.375274 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ff105905-d7be-4106-ba66-bb69a8db5313) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTestJSON-Volume-747665335',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=73066e68-2746-4d0a-9bd5-037246d03f73,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='ea1927109a494e6a9c8edae7c8ea87cd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:29Z,use_quota=True,user_id='49d724c1b2454a1c988e4c8b5d361b35',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.380640 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a0fc83aa-2555-4cc2-b7a8-69f8fe826a8b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.385397 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a0fc83aa-2555-4cc2-b7a8-69f8fe826a8b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.390487 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6bcbdc96-0b48-4e0d-b21a-f544f239775f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.394472 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6bcbdc96-0b48-4e0d-b21a-f544f239775f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '73066e68-2746-4d0a-9bd5-037246d03f73', 'volume_name': 'volume-73066e68-2746-4d0a-9bd5-037246d03f73', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.400278 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d954c6ad-b2c9-4f2d-9d05-88d0470355ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.404124 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d954c6ad-b2c9-4f2d-9d05-88d0470355ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:29.408800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2e842ee-8436-4a74-b543-b7bcd57e7075) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:29.412911 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Volume 73066e68-2746-4d0a-9bd5-037246d03f73: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-73066e68-2746-4d0a-9bd5-037246d03f73', 'volume_size': 1} May 14 21:40:29.412911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] creating volume 'volume-73066e68-2746-4d0a-9bd5-037246d03f73' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:29.412911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:30.170828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b6c7de54-3c8e-4da5-aa0b-bc6e9ae68838 tempest-VolumeRetypeWithoutMigrationTest-657059995 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:30.202357 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b6c7de54-3c8e-4da5-aa0b-bc6e9ae68838 tempest-VolumeRetypeWithoutMigrationTest-657059995 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:30.246222 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-b6c7de54-3c8e-4da5-aa0b-bc6e9ae68838 tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:30.254214 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b6c7de54-3c8e-4da5-aa0b-bc6e9ae68838 tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Retype volume completed successfully. May 14 21:40:30.709150 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:30.709558 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:30.713501 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2e842ee-8436-4a74-b543-b7bcd57e7075) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-73066e68-2746-4d0a-9bd5-037246d03f73', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:30.718203 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (753bd606-c6d7-4933-a341-9943b8214c48) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:30.816065 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Volume volume-73066e68-2746-4d0a-9bd5-037246d03f73 (73066e68-2746-4d0a-9bd5-037246d03f73): created successfully May 14 21:40:30.819849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (753bd606-c6d7-4933-a341-9943b8214c48) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:30.825598 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Flow 'volume_create_manager' (21090453-3e5a-4956-8304-d73c9bd4bfdc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:30.924568 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-39ccddcb-297f-49d1-b679-9381469ce894 req-b8518ebc-ea74-45d1-9bdf-691f4498a388 tempest-VolumesGetTestJSON-1485306589 None] Created volume successfully. May 14 21:40:30.992708 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:31.001695 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:31.001695 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (742c254a-64c7-4007-a3d7-99aac53856bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:31.002731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2946a115-8145-43d4-ba29-44d89f9aa74c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:31.111697 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Volume volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b (7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b): created successfully May 14 21:40:31.116051 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2946a115-8145-43d4-ba29-44d89f9aa74c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:31.122416 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Flow 'volume_create_manager' (1cd65996-e86f-4464-9359-1a37938a3a50) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:31.198750 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-86ae6f36-97ba-4995-bc6b-facf195f043f tempest-VolumesGetTest-1279583530 None] Created volume successfully. May 14 21:40:31.239749 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Lock "b'cinder-28ef052f-001b-41b0-80a6-22122e8dd2d0-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:31.395185 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:31.924557 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:31.930044 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Volume volume-28ef052f-001b-41b0-80a6-22122e8dd2d0 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:32.239293 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] deleting rbd volume volume-28ef052f-001b-41b0-80a6-22122e8dd2d0 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:32.252950 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] Lock "b'cinder-73066e68-2746-4d0a-9bd5-037246d03f73-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:32.404788 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:32.678939 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:32.684116 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] Volume volume-73066e68-2746-4d0a-9bd5-037246d03f73 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:32.833301 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] deleting rbd volume volume-73066e68-2746-4d0a-9bd5-037246d03f73 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:32.846076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:32.942678 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:33.271558 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:33.275454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] Volume volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:33.530524 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:36.765104 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] Created reservations ['e787d3c5-5197-413c-ad17-321001c7bbe3', 'd40198e5-b2e4-429f-9b71-2bfd087fcbd8', '64f0e615-b356-40af-925f-79befa77cb62', '82662461-5dbf-4a60-b6dc-f0cec81eeee1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:36.886215 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] Terminate volume connection completed successfully. May 14 21:40:36.905160 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-84a90b1c-0963-413a-bb4f-7d5012d604dc tempest-VolumesBackupsV39Test-500200030 None] Remove volume export completed successfully. May 14 21:40:37.039467 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:37.075700 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:37.105300 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:37.109310 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] Deleted volume successfully. May 14 21:40:37.264745 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b27b3b0b-d681-45c5-a0b2-5339dd342ba5 tempest-AbsoluteLimitsTests-1102377737 None] Lock "b'cinder-74bb8a05-2411-40ba-acdf-f6a7442d9843-delete_volume'" released by "delete_volume" :: held 9.568s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:40.473215 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Deprecated: Default quota for resource: volumes_volume-type-2 is set by the default quota flag: quota_volumes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. May 14 21:40:40.474121 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Deprecated: Default quota for resource: gigabytes_volume-type-2 is set by the default quota flag: quota_gigabytes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. May 14 21:40:40.564544 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Created reservations ['6b273388-40a9-45b4-a57c-0a2596fcb903', '5bb1a8ac-ab54-437e-960d-7d796997208d', '030b8b7f-87b8-4e6b-a0dc-86fd8820cd25', '13e4c1d9-8872-41fb-ab9c-976f15e164bb'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:40.889194 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:40.938754 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:40.979677 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:40.985944 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Deleted volume successfully. May 14 21:40:41.129540 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-f2e665cc-e2da-43ae-b498-54fd149f93ee tempest-VolumeRetypeWithoutMigrationTest-657059995 None] Lock "b'cinder-28ef052f-001b-41b0-80a6-22122e8dd2d0-delete_volume'" released by "delete_volume" :: held 9.890s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:41.325238 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] Created reservations ['0fc1e598-f00c-4539-8bd3-d7ac92cf4be5', '7130ed92-d0f1-4cff-9190-0df54d03b019', '0734d373-345f-434f-960e-1bec66296ff4', 'bc7dedea-7ca8-4ac5-b498-241aa9889373'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:41.392337 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] Created reservations ['c7acf058-79ba-40ba-8a8c-390db9e00994', '227690fa-c6e4-40d9-adac-e1ee8f5d0309', 'b6af26b9-2757-40b1-93b1-196a84e5bdbe', '64ffff3d-9127-4038-8c05-bd0cb06b38bf'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:41.512330 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:41.540522 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:41.592831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:41.601421 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:40:41.717396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-b0390497-15ec-47f4-b5a3-4f0176f78bbc tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-7761e0c8-99bf-4383-a8ef-b7ec3ad23e7b-delete_volume'" released by "delete_volume" :: held 8.871s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:41.753668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:41.792685 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:41.892312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:41.905179 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] Deleted volume successfully. May 14 21:40:41.980542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-5b019998-6fcd-43d8-ad1a-5b6164939820 req-cb780327-c7b5-47f2-b8d8-06340318bf51 tempest-VolumesGetTestJSON-1485306589 None] Lock "b'cinder-73066e68-2746-4d0a-9bd5-037246d03f73-delete_volume'" released by "delete_volume" :: held 9.725s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:42.000081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:42.020457 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Flow 'volume_create_manager' (eec4faac-df22-46a9-ac99-f507e0b3178c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:42.030476 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6942ac3a-f240-4cd8-8363-0166412656fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:42.071718 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6942ac3a-f240-4cd8-8363-0166412656fc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-108644161',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8e223e25-283c-4416-8096-5d0218c12613,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9ef324a7ffd642f08d0c23153a225daa',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:42Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:42Z,use_quota=True,user_id='5745c928a98f441d9000ff599dfdc274',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:42.079361 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0fc4de25-ac9f-4fab-8d7c-486a097c4e5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:42.085295 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0fc4de25-ac9f-4fab-8d7c-486a097c4e5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:42.092273 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6dcd493c-903e-488a-ad28-c624ad0375de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:42.097107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6dcd493c-903e-488a-ad28-c624ad0375de) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8e223e25-283c-4416-8096-5d0218c12613', 'volume_name': 'volume-8e223e25-283c-4416-8096-5d0218c12613', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:42.104155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4450a54f-41cd-458a-9c49-95c0be990e25) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:42.109000 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4450a54f-41cd-458a-9c49-95c0be990e25) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:42.117126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7057337a-6409-4428-aaef-ac5a6746bcde) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:42.118519 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Volume 8e223e25-283c-4416-8096-5d0218c12613: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8e223e25-283c-4416-8096-5d0218c12613', 'volume_size': 1} May 14 21:40:42.118696 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] creating volume 'volume-8e223e25-283c-4416-8096-5d0218c12613' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:42.119506 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:42.419077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:42.542875 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:42.842622 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:43.244871 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:43.374268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:43.378805 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:43.380082 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7057337a-6409-4428-aaef-ac5a6746bcde) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8e223e25-283c-4416-8096-5d0218c12613', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:43.385271 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aeba175c-1a21-4407-a41f-394d54e54ab9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:43.459827 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Volume volume-8e223e25-283c-4416-8096-5d0218c12613 (8e223e25-283c-4416-8096-5d0218c12613): created successfully May 14 21:40:43.464216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aeba175c-1a21-4407-a41f-394d54e54ab9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:43.469819 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Flow 'volume_create_manager' (eec4faac-df22-46a9-ac99-f507e0b3178c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:43.613930 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fd252d18-2d9a-4eb3-947c-a66414b0cb2a tempest-VolumesMetadataTest-497296701 None] Created volume successfully. May 14 21:40:45.288926 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] Lock "b'cinder-8e223e25-283c-4416-8096-5d0218c12613-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:45.500187 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:45.751134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:45.751725 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] Volume volume-8e223e25-283c-4416-8096-5d0218c12613 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:46.022292 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] deleting rbd volume volume-8e223e25-283c-4416-8096-5d0218c12613 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:46.183419 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:46.203766 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Flow 'volume_create_manager' (5d0404b5-445a-4635-9338-eb855563beff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:46.210379 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ffdb224f-5d8a-494a-aeb1-af7fa9b863f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.225788 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ffdb224f-5d8a-494a-aeb1-af7fa9b863f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-217602726',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0081333d-662b-4fe5-9d3c-2827b4734492,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='a646ef9c31fd4125ad0c8196768a072e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:46Z,use_quota=True,user_id='ac7356d0eff7427d873ded6b8dff90c5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.231269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b96c3589-a0ec-4afc-b2a2-128557f95762) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.235498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b96c3589-a0ec-4afc-b2a2-128557f95762) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.240775 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5725b293-89fc-4f50-921d-ea72ce7966fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.245911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5725b293-89fc-4f50-921d-ea72ce7966fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0081333d-662b-4fe5-9d3c-2827b4734492', 'volume_name': 'volume-0081333d-662b-4fe5-9d3c-2827b4734492', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.250878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (15b92071-8cd4-489d-b5b5-ee4d8a2ce7d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.254795 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (15b92071-8cd4-489d-b5b5-ee4d8a2ce7d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.259806 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7bdb633e-1173-4127-9d50-579a9617175e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.260885 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Volume 0081333d-662b-4fe5-9d3c-2827b4734492: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0081333d-662b-4fe5-9d3c-2827b4734492', 'volume_size': 1} May 14 21:40:46.261278 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] creating volume 'volume-0081333d-662b-4fe5-9d3c-2827b4734492' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:46.261945 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:46.605835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:46.625223 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Flow 'volume_create_manager' (728aee19-6c59-4e94-a488-1c9a8417edc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:46.631977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3739c514-6045-4fd4-8228-f05891301c6a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.648220 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3739c514-6045-4fd4-8228-f05891301c6a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesDeleteCascade-Volume-525269605',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=03fd5a86-fb73-496d-aff1-8194e17894cf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='eaa012bd5cf0437f9a1d0a524871726f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:46Z,use_quota=True,user_id='1b8033dfa10f4a638f4136ed3449eea0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.654530 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c0c2b870-28c9-47df-9694-40fb4f43e908) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.658730 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c0c2b870-28c9-47df-9694-40fb4f43e908) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.664312 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff7637f0-c5e4-4fee-9b98-c1261735af7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.668669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff7637f0-c5e4-4fee-9b98-c1261735af7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '03fd5a86-fb73-496d-aff1-8194e17894cf', 'volume_name': 'volume-03fd5a86-fb73-496d-aff1-8194e17894cf', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.674162 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (109d122a-7b1b-470c-b453-c646cc0fdf2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.678729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (109d122a-7b1b-470c-b453-c646cc0fdf2a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:46.683723 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (75dcd178-9d13-4fbb-b292-d299bec0c62a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:46.685101 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Volume 03fd5a86-fb73-496d-aff1-8194e17894cf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-03fd5a86-fb73-496d-aff1-8194e17894cf', 'volume_size': 1} May 14 21:40:46.685231 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] creating volume 'volume-03fd5a86-fb73-496d-aff1-8194e17894cf' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:46.685885 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:47.328064 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:47.328443 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:47.339821 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7bdb633e-1173-4127-9d50-579a9617175e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0081333d-662b-4fe5-9d3c-2827b4734492', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:47.344788 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (41c6f5d6-e320-470b-82bd-cc37102f59b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:47.439773 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Volume volume-0081333d-662b-4fe5-9d3c-2827b4734492 (0081333d-662b-4fe5-9d3c-2827b4734492): created successfully May 14 21:40:47.447467 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (41c6f5d6-e320-470b-82bd-cc37102f59b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:47.456049 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Flow 'volume_create_manager' (5d0404b5-445a-4635-9338-eb855563beff) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:47.551332 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6c709ea8-c144-46e9-8085-f1dcc77af3a6 tempest-VolumesTestJSON-1883546231 None] Created volume successfully. May 14 21:40:47.826392 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:47.827533 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:47.838087 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (75dcd178-9d13-4fbb-b292-d299bec0c62a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-03fd5a86-fb73-496d-aff1-8194e17894cf', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:47.843233 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (83f36aab-b663-4b10-8c1a-7499430aed29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:47.935077 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Volume volume-03fd5a86-fb73-496d-aff1-8194e17894cf (03fd5a86-fb73-496d-aff1-8194e17894cf): created successfully May 14 21:40:47.935773 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (83f36aab-b663-4b10-8c1a-7499430aed29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:47.941446 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Flow 'volume_create_manager' (728aee19-6c59-4e94-a488-1c9a8417edc3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:48.049646 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-457960b9-32b4-4936-a383-167d7a5b055d tempest-VolumesDeleteCascade-802935524 None] Created volume successfully. May 14 21:40:49.010704 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:49.043425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Flow 'volume_create_manager' (d42c54c9-666a-4d45-b7ae-b2f8d44569e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:49.059150 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1f76be8-9767-4ad2-ab5f-f641736b4491) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:49.077117 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1f76be8-9767-4ad2-ab5f-f641736b4491) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1900737035',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a49a968d-ae9a-42e3-9f36-1a7388b3a57a,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='a646ef9c31fd4125ad0c8196768a072e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:49Z,use_quota=True,user_id='ac7356d0eff7427d873ded6b8dff90c5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:49.083154 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (750c3c5e-5579-4788-abf6-382625a4905a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:49.088349 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (750c3c5e-5579-4788-abf6-382625a4905a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:49.093801 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3905cf0b-8f77-4a52-a318-07807c95db07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:49.098003 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3905cf0b-8f77-4a52-a318-07807c95db07) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a49a968d-ae9a-42e3-9f36-1a7388b3a57a', 'volume_name': 'volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:49.104720 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6bd979e5-f70b-4a75-bf42-a3b67abe2e52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:49.109767 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6bd979e5-f70b-4a75-bf42-a3b67abe2e52) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:49.114897 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a1de7398-f8f7-49f5-84c3-5445892d29b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:49.116085 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Volume a49a968d-ae9a-42e3-9f36-1a7388b3a57a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a', 'volume_size': 1} May 14 21:40:49.116479 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] creating volume 'volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:49.117217 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:49.413638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2bb3bb1d-9ddb-4150-98d2-efc56c82dc98 tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:50.283049 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] Lock "b'cinder-af0dc711-750a-44a0-b81b-0754b812e053-delete_volume'" acquired by "delete_volume" :: waited 0.026s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:50.394824 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:53.286641 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] Volume volume-af0dc711-750a-44a0-b81b-0754b812e053 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:53.734366 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] deleting rbd volume volume-af0dc711-750a-44a0-b81b-0754b812e053 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:53.745752 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:40:53.746513 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:40:53.771040 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a1de7398-f8f7-49f5-84c3-5445892d29b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:53.783092 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ba253593-d50a-477a-925a-e30607529442) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:53.788797 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:40:54.010260 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] deleting parent snapshot volume-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:40:58.110180 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Volume volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a (a49a968d-ae9a-42e3-9f36-1a7388b3a57a): created successfully May 14 21:40:58.116521 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ba253593-d50a-477a-925a-e30607529442) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:58.120661 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Flow 'volume_create_manager' (d42c54c9-666a-4d45-b7ae-b2f8d44569e7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:58.192460 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] Created reservations ['f6983ea3-bf00-4052-af3e-8472082d60f0', '09af448d-bca3-42d6-9869-9ae537bd2571', '1708c6c5-333a-4747-ac78-db5964a8dbb0', '580df9a8-6187-4311-9f3d-5d114347f9a2'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:58.220104 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4b8c4740-d308-4b6c-9abc-92bf6eec5157 tempest-VolumesTestJSON-1883546231 None] Created volume successfully. May 14 21:40:58.223631 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] Created reservations ['504c3734-0689-44f0-b882-dba548878681', 'b1b914e7-c035-4f2a-96f2-856d375431fa', 'e9d9d83a-4375-430d-b84b-3f33d07da1d3', '7c7e3bad-edd6-4689-a9fa-2c64dfc884d0'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:58.456235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:58.504912 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:58.576487 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:58.581881 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:40:58.686765 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:58.750796 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:58.814068 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:40:58.824867 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] Deleted volume successfully. May 14 21:40:58.831707 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d3e7e0b5-87bb-4b01-b471-9dc3cc627529 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-45e27f5f-4c1f-45aa-b6b1-1ea7e8c05e50-delete_volume'" released by "delete_volume" :: held 16.411s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:58.836228 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-cf7210ee-929c-4ef8-a22e-825b560ca0ff-delete_volume'" acquired by "delete_volume" :: waited 0.017s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:40:58.951095 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ff59bfe1-09fa-4fda-b2ca-c41204b77a67 tempest-VolumesMetadataTest-497296701 None] Lock "b'cinder-8e223e25-283c-4416-8096-5d0218c12613-delete_volume'" released by "delete_volume" :: held 13.662s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:40:58.966928 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:59.190872 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:40:59.191587 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] Volume volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:40:59.353683 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] deleting rbd volume volume-cf7210ee-929c-4ef8-a22e-825b560ca0ff {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:40:59.372277 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2bb3bb1d-9ddb-4150-98d2-efc56c82dc98 tempest-VolumesDeleteCascade-802935524 None] Create snapshot completed successfully May 14 21:40:59.503657 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:40:59.524313 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Flow 'volume_create_manager' (0098b332-16fd-4e30-91ac-fa0d8a88dde9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:40:59.531725 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (88535214-6595-41ee-af4d-5c5ba12abe27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:59.537572 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] Created reservations ['90b28930-6160-4140-8f99-8664bdef61cf', 'c8ba2319-5972-4f27-8109-55e0bf2d4d25', 'f783c2d1-ec58-46ca-b42d-0f597001812b', 'b62cf18d-7e63-49de-90ed-e5f7fac74ce3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:40:59.564328 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (88535214-6595-41ee-af4d-5c5ba12abe27) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:40:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1192155142',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1f8e3a83-ebb1-4ff9-a436-767fd40be90c,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='a646ef9c31fd4125ad0c8196768a072e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:40:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:40:59Z,use_quota=True,user_id='ac7356d0eff7427d873ded6b8dff90c5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:59.573494 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ae6fa8af-6fa5-475d-8bf6-b124663ee7c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:59.577906 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ae6fa8af-6fa5-475d-8bf6-b124663ee7c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:59.585192 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6ea01093-abfd-47b7-956f-120389add134) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:59.591407 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6ea01093-abfd-47b7-956f-120389add134) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1f8e3a83-ebb1-4ff9-a436-767fd40be90c', 'volume_name': 'volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:59.596761 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d469fe84-627d-4301-b591-f27f987631fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:59.601049 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d469fe84-627d-4301-b591-f27f987631fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:40:59.606911 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9b6ec4bb-2796-4a43-9ab1-f298272ceb3a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:40:59.610743 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Volume 1f8e3a83-ebb1-4ff9-a436-767fd40be90c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c', 'volume_size': 1} May 14 21:40:59.611290 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] creating volume 'volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:40:59.612902 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:59.940443 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:40:59.987709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:00.031721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:00.037597 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] Deleted volume successfully. May 14 21:41:00.184732 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-485385f7-fa25-4657-97fb-c5598ef3929a tempest-VolumesBackupsV39Test-500200030 None] Lock "b'cinder-af0dc711-750a-44a0-b81b-0754b812e053-delete_volume'" released by "delete_volume" :: held 9.902s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:00.393197 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3740e39c-37f5-4da2-b2b3-6934e3151dd1 tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:00.847184 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:41:00.847811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:41:00.853495 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9b6ec4bb-2796-4a43-9ab1-f298272ceb3a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:00.861156 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (75210098-400d-4c30-b31f-b72a83dc08c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:00.955299 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Volume volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c (1f8e3a83-ebb1-4ff9-a436-767fd40be90c): created successfully May 14 21:41:00.957438 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (75210098-400d-4c30-b31f-b72a83dc08c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:00.965610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Flow 'volume_create_manager' (0098b332-16fd-4e30-91ac-fa0d8a88dde9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:01.043618 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f5eb86bc-57ed-432e-8c04-abba019e5ccf tempest-VolumesTestJSON-1883546231 None] Created volume successfully. May 14 21:41:02.683119 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] Lock "b'cinder-1f8e3a83-ebb1-4ff9-a436-767fd40be90c-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:02.781927 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:03.073226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:41:03.077118 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] Volume volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:41:03.374693 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] deleting rbd volume volume-1f8e3a83-ebb1-4ff9-a436-767fd40be90c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:41:03.519079 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:41:03.556801 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Flow 'volume_create_manager' (76f271df-8293-4fc7-ae0d-534401a27c55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:03.564500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbf7b3a6-e7f5-4b2b-87bf-32d1f739ee6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:03.583886 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbf7b3a6-e7f5-4b2b-87bf-32d1f739ee6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:41:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesNegativeTest-Volume-245143988',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=559e64fc-8aa1-44c2-98e6-8f8a1a7566cc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5f29dab7e48a4a9bba08ffffacbe5c8a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:41:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:41:03Z,use_quota=True,user_id='b61e5fd331864e21b5c6aa52dde37bec',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:03.591034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (61b7353e-d43e-47c0-b08f-430f6c63f8c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:03.595320 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (61b7353e-d43e-47c0-b08f-430f6c63f8c6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:03.600678 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (63cb2127-8ab0-40af-8bdf-3b54962df330) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:03.605918 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (63cb2127-8ab0-40af-8bdf-3b54962df330) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '559e64fc-8aa1-44c2-98e6-8f8a1a7566cc', 'volume_name': 'volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:03.611509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8a44b8df-5de0-4080-a8b1-d1fb1bb17a4b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:03.616571 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8a44b8df-5de0-4080-a8b1-d1fb1bb17a4b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:03.622651 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (afa4ebfc-d08b-413b-b706-27b6b362f6ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:03.623905 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Volume 559e64fc-8aa1-44c2-98e6-8f8a1a7566cc: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc', 'volume_size': 1} May 14 21:41:03.624363 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] creating volume 'volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:41:03.625127 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:03.722526 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:41:03.723264 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:03.767715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:03.806220 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:04.125214 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:41:04.145216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Flow 'volume_create_manager' (08574ff7-18d4-4cb6-b719-9c4477cbb753) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:04.152667 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e6228b00-f3be-4a40-94cc-787e7361e1bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:04.168343 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e6228b00-f3be-4a40-94cc-787e7361e1bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:41:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotNegativeTestJSON-Volume-1280890739',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=68ae59c7-a5d5-46a1-98c0-7a020f158a58,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c4f2af00e6df44ca8d789add57208278',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:41:04Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:41:04Z,use_quota=True,user_id='836726f056a74d80ab7a04cb60268d9d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:04.173866 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5f064c8a-e5d4-494e-b468-07c4fe535994) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:04.178711 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5f064c8a-e5d4-494e-b468-07c4fe535994) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:04.183771 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (75646e72-c2ba-4bee-a0f2-039ea5fce756) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:04.187858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (75646e72-c2ba-4bee-a0f2-039ea5fce756) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '68ae59c7-a5d5-46a1-98c0-7a020f158a58', 'volume_name': 'volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58', 'volume_size': 2}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:04.192807 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (24a398e4-c43a-4a95-9fcb-8759a0f6ba2b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:04.197442 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (24a398e4-c43a-4a95-9fcb-8759a0f6ba2b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:04.202218 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f6a95d14-c15e-4163-8c65-38ce37e02fda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:04.203348 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Volume 68ae59c7-a5d5-46a1-98c0-7a020f158a58: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58', 'volume_size': 2} May 14 21:41:04.205532 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] creating volume 'volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:41:04.205532 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:04.712124 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3740e39c-37f5-4da2-b2b3-6934e3151dd1 tempest-VolumesDeleteCascade-802935524 None] Create snapshot completed successfully May 14 21:41:04.844515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:41:04.848202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:41:04.848800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (afa4ebfc-d08b-413b-b706-27b6b362f6ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:04.855202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c2b01021-6cab-4817-b058-bf22cac0f67f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:04.929358 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Volume volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc (559e64fc-8aa1-44c2-98e6-8f8a1a7566cc): created successfully May 14 21:41:04.933944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c2b01021-6cab-4817-b058-bf22cac0f67f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:04.940982 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Flow 'volume_create_manager' (76f271df-8293-4fc7-ae0d-534401a27c55) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:05.021540 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e033f3b3-40b3-48af-b297-7658beb8698f tempest-VolumesNegativeTest-342437217 None] Created volume successfully. May 14 21:41:05.340386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:41:05.341398 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:41:05.344986 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f6a95d14-c15e-4163-8c65-38ce37e02fda) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58', 'volume_size': 2}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:05.350046 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (398dd852-c022-4eba-90c4-9435a929f11d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:05.398269 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Volume volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58 (68ae59c7-a5d5-46a1-98c0-7a020f158a58): created successfully May 14 21:41:05.402108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (398dd852-c022-4eba-90c4-9435a929f11d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:05.418350 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Flow 'volume_create_manager' (08574ff7-18d4-4cb6-b719-9c4477cbb753) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:05.467875 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-af25a5f7-8f2f-4e6f-b38f-8d0e0dd3e4d3 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Created volume successfully. May 14 21:41:05.729943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Lock "b'cinder-03fd5a86-fb73-496d-aff1-8194e17894cf-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:05.840709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Performing cascade delete. {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/manager.py:964}} May 14 21:41:05.864335 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Lock "b'cinder-7f6eb6ac-47dc-4705-9831-f124e1284177-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:05.878686 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:06.398786 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] Created reservations ['b1858f7a-6ece-42ed-9c53-942e11dfa467', '404dbdff-d145-46f0-a681-c86761f93bc9', '7533eab2-a438-4e68-b819-16fe69789b44', '14bacd8a-2961-4c1a-9881-608f64fd5ff7'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:06.667762 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:06.693185 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:06.718889 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:06.730309 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] Deleted volume successfully. May 14 21:41:06.810722 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-f8d0871f-cf46-4dd5-b258-85412e30c441 tempest-VolumesGetTest-1279583530 None] Lock "b'cinder-cf7210ee-929c-4ef8-a22e-825b560ca0ff-delete_volume'" released by "delete_volume" :: held 7.974s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:06.847294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45cdb4a4-f183-4554-bdd6-00e72b4df4e4 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:10.313329 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Created reservations ['08f66964-b505-430d-a10b-9ccbf2e29d89', 'f315af9a-b102-4694-aa4c-208b3e77c77e', '7f065f09-b0d0-443f-9b24-36929aa825bb', 'cdce0a45-e78a-497b-a212-fe544adb759c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:10.622037 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] Created reservations ['2820ed93-3f1f-4828-8155-17ca4b9dfe2c', '6c3fe99c-dca2-4902-b429-02bd4c3d259f', '8b784014-cc1e-4fa7-8fa9-50377a433d01', '1fb4b855-2872-456f-86ac-bf26bf72811f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:10.774896 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Delete snapshot completed successfully. May 14 21:41:10.781488 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Lock "b'cinder-7f6eb6ac-47dc-4705-9831-f124e1284177-delete_snapshot'" released by "delete_snapshot" :: held 4.917s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:10.815902 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Lock "b'cinder-b3b61a1d-3f46-4381-9450-3f1aba0af24a-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.030s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:10.837489 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:11.055345 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-45cdb4a4-f183-4554-bdd6-00e72b4df4e4 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Create snapshot completed successfully May 14 21:41:11.192874 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:11.301211 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:11.348286 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:11.354416 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] Deleted volume successfully. May 14 21:41:11.460863 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d6ac24e8-cfe2-461d-9d33-684bf885d2de tempest-VolumesTestJSON-1883546231 None] Lock "b'cinder-1f8e3a83-ebb1-4ff9-a436-767fd40be90c-delete_volume'" released by "delete_volume" :: held 8.777s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:11.621009 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] Lock "b'cinder-a49a968d-ae9a-42e3-9f36-1a7388b3a57a-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:11.735115 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:12.011675 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:41:12.011675 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] Volume volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:41:12.271275 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] deleting rbd volume volume-a49a968d-ae9a-42e3-9f36-1a7388b3a57a {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:41:12.285386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7f0a8dbc-f168-416a-bfb1-b960c2fe6499 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Lock "b'cinder-d97d8e5b-b52b-4d9a-851c-a3f4a20b6f74-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:12.286850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7f0a8dbc-f168-416a-bfb1-b960c2fe6499 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:14.181441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Created reservations ['9e10d5cf-6e9b-4d4e-9982-d7560a12282e', 'f92ba49b-ace8-4227-91ed-a961c7206abf', '26e74542-01bc-4e8f-aed8-df3b10092a77', '732bee32-6fba-4940-abb2-624ea442aea3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:14.379403 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Delete snapshot completed successfully. May 14 21:41:14.386343 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Lock "b'cinder-b3b61a1d-3f46-4381-9450-3f1aba0af24a-delete_snapshot'" released by "delete_snapshot" :: held 3.571s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:14.389395 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Snapshots deleted, issuing volume delete {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/manager.py:979}} May 14 21:41:14.389395 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:14.654558 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:41:14.655160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Volume volume-03fd5a86-fb73-496d-aff1-8194e17894cf is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:41:14.776993 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] deleting rbd volume volume-03fd5a86-fb73-496d-aff1-8194e17894cf {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:41:17.375096 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7f0a8dbc-f168-416a-bfb1-b960c2fe6499 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Created reservations ['65a0aab1-141b-476e-80a0-a9dd76929ad7', 'bae4cbe9-74e0-4980-b641-ed2107ec0753', '711126f0-286e-4e36-b1b0-79f38f60c0b8', '210b4b0a-bdf0-4da8-a801-520d7036a7af'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:17.562128 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7f0a8dbc-f168-416a-bfb1-b960c2fe6499 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Delete snapshot completed successfully. May 14 21:41:17.568895 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7f0a8dbc-f168-416a-bfb1-b960c2fe6499 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Lock "b'cinder-d97d8e5b-b52b-4d9a-851c-a3f4a20b6f74-delete_snapshot'" released by "delete_snapshot" :: held 5.284s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:18.595141 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Lock "b'cinder-68ae59c7-a5d5-46a1-98c0-7a020f158a58-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:18.712841 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:18.951096 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:41:18.951548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Volume volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:41:19.130085 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] deleting rbd volume volume-68ae59c7-a5d5-46a1-98c0-7a020f158a58 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:41:19.792610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] Created reservations ['94bdebdf-3a4e-4e6f-a528-fe0827988ef1', '73297f45-09f9-4e3e-9141-eb96191a01c9', '62244617-2ab1-4910-a287-afe3b75eef21', '9008a9fd-4e44-49ea-b70d-19fbde2903fb'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:19.985309 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:20.007520 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:20.032655 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:20.036937 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] Deleted volume successfully. May 14 21:41:20.134351 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-57b722f5-f702-4741-8cc4-5827477f60bb tempest-VolumesTestJSON-1883546231 None] Lock "b'cinder-a49a968d-ae9a-42e3-9f36-1a7388b3a57a-delete_volume'" released by "delete_volume" :: held 8.513s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:20.309965 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] Lock "b'cinder-0081333d-662b-4fe5-9d3c-2827b4734492-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:20.404252 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:20.564334 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:41:20.564334 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] Volume volume-0081333d-662b-4fe5-9d3c-2827b4734492 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:41:20.787892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] deleting rbd volume volume-0081333d-662b-4fe5-9d3c-2827b4734492 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:41:22.032500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Created reservations ['8f426e18-aaa0-4be1-9763-c11bf13d633a', 'd4a5ea2d-9590-4323-a322-633a96954747', '7f40baf6-70a0-4bf2-9ba9-cacba0f50bd6', 'f7471921-b31b-4659-9ee0-c5f5cf3ada0c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:22.192742 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:22.248690 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:22.272527 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:22.277197 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Deleted volume successfully. May 14 21:41:22.420858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-e51aeacd-df49-4ccf-8175-5c4338278e0a tempest-VolumesDeleteCascade-802935524 None] Lock "b'cinder-03fd5a86-fb73-496d-aff1-8194e17894cf-delete_volume'" released by "delete_volume" :: held 16.691s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:25.894305 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Created reservations ['73ce3bc5-fbd0-421a-9a30-bcb6f72ebd22', 'bf1334e8-7d5c-497c-8372-c3366179cd27', '3666c5d6-ff65-41e9-b12d-dc2ae1fed401', 'f7fed5d3-9db1-482c-86cf-bbc9f6fb563a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:26.221980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:26.262366 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:26.307030 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:26.317036 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Deleted volume successfully. May 14 21:41:26.453688 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-939c0e57-5246-4170-9115-c3e3a6d76be9 tempest-VolumesSnapshotNegativeTestJSON-143892314 None] Lock "b'cinder-68ae59c7-a5d5-46a1-98c0-7a020f158a58-delete_volume'" released by "delete_volume" :: held 7.858s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:26.901317 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] Created reservations ['8630b57f-e9a2-4bd0-b878-7d908f09134d', '817380c3-b5c9-4577-a02e-b7b1ecf4b798', '35cb571e-e406-4639-bbdc-3bc2e5ade52d', 'a6c08557-351e-47d9-8fc7-5ee6afa5e8f1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:27.250353 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:27.325987 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:27.459597 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:27.469675 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] Deleted volume successfully. May 14 21:41:27.572523 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-93cdf969-f399-43ba-8429-a34c09b0d130 tempest-VolumesTestJSON-1883546231 None] Lock "b'cinder-0081333d-662b-4fe5-9d3c-2827b4734492-delete_volume'" released by "delete_volume" :: held 7.262s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:29.358950 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:41:29.401862 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Flow 'volume_create_manager' (c58d8ecd-bc7d-4cba-b864-75b7e0fbbeab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:29.408431 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2d2b66a5-a95e-4025-9e01-bca1da665c33) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:29.431119 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2d2b66a5-a95e-4025-9e01-bca1da665c33) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:41:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-646466756',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='02f7a491fa1c45c896f1af07217a303a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:41:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:41:29Z,use_quota=True,user_id='9b156d4dd0064d63b2809fa6327ed8aa',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:29.436831 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7e178d60-d00b-42c9-8ddf-07d56a6fb124) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:29.441047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7e178d60-d00b-42c9-8ddf-07d56a6fb124) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:29.447425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (798c1fec-651e-43f4-804c-5bea7cd22c03) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:29.452209 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (798c1fec-651e-43f4-804c-5bea7cd22c03) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b', 'volume_name': 'volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:29.458240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e4a2890d-5e5c-469f-b494-f0c7ef316c75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:29.463857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e4a2890d-5e5c-469f-b494-f0c7ef316c75) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:29.468648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (440dae0c-191c-45db-ad76-d8283a1e3f7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:29.471956 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Volume c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b', 'volume_size': 1} May 14 21:41:29.471956 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] creating volume 'volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:41:29.471956 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:30.194068 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:41:30.194698 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:41:30.206135 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (440dae0c-191c-45db-ad76-d8283a1e3f7f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:30.217337 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (511a2f1c-0c7f-4295-af75-d80e0d697cd6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:41:30.299387 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Volume volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b (c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b): created successfully May 14 21:41:30.304160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (511a2f1c-0c7f-4295-af75-d80e0d697cd6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:41:30.310904 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Flow 'volume_create_manager' (c58d8ecd-bc7d-4cba-b864-75b7e0fbbeab) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:41:30.384926 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2b508d20-509e-4a9c-91d3-a9755a2ce12f tempest-VolumesActionsTest-1847628510 None] Created volume successfully. May 14 21:41:34.093291 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] Lock "b'cinder-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:34.207879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:34.385957 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:41:34.386394 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] Volume volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:41:34.466555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] deleting rbd volume volume-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:41:38.398414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] Created reservations ['04174cf1-0d58-4df3-a02a-346381e4d574', 'ed70109e-93c5-4278-957c-02de07f536d7', '6d94f131-1b8f-4174-845a-7b8da9a2c55b', 'b98137af-6e74-434f-b771-979844101f07'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:41:38.592323 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:38.623646 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:41:38.655376 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:41:38.659539 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] Deleted volume successfully. May 14 21:41:38.794169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-46b49861-e7f5-448d-b15b-815461abab33 tempest-VolumesNegativeTest-342437217 None] Lock "b'cinder-559e64fc-8aa1-44c2-98e6-8f8a1a7566cc-delete_volume'" released by "delete_volume" :: held 4.704s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:51.655815 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3590f088-e786-4dc1-8caa-5ba098eb2b3c tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b'" acquired by "attach_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:51.918152 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3590f088-e786-4dc1-8caa-5ba098eb2b3c tempest-VolumesActionsTest-1847628510 None] Attaching volume c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b to instance 30cccea2-386c-432b-ae51-d41981ed64ce at mountpoint /dev/sdb on host None. May 14 21:41:52.072861 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3590f088-e786-4dc1-8caa-5ba098eb2b3c tempest-VolumesActionsTest-1847628510 None] Attach volume completed successfully. May 14 21:41:52.088624 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3590f088-e786-4dc1-8caa-5ba098eb2b3c tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b'" released by "attach_volume" :: held 0.433s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:41:52.315943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8b9cd3a5-8fa2-4e39-9693-6d28a48f3d13 tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-detach_volume'" acquired by "detach_volume" :: waited 0.028s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:41:52.318693 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8b9cd3a5-8fa2-4e39-9693-6d28a48f3d13 tempest-VolumesActionsTest-1847628510 None] Detaching volume c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b from instance 30cccea2-386c-432b-ae51-d41981ed64ce. May 14 21:41:52.571828 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8b9cd3a5-8fa2-4e39-9693-6d28a48f3d13 tempest-VolumesActionsTest-1847628510 None] Detach volume completed successfully. May 14 21:41:52.577555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-8b9cd3a5-8fa2-4e39-9693-6d28a48f3d13 tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-detach_volume'" released by "detach_volume" :: held 0.267s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:03.728985 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:42:03.729563 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:03.794727 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:03.876241 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:42:15.981017 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-45eaa862-e963-4b91-b362-9749d3014ab5 tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b'" acquired by "attach_volume" :: waited 0.023s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:16.248239 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-45eaa862-e963-4b91-b362-9749d3014ab5 tempest-VolumesActionsTest-1847628510 None] Attaching volume c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b to instance 3ee751a8-3bfb-4b61-8117-6f8e05f6cd88 at mountpoint /dev/sdb on host None. May 14 21:42:16.363100 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-45eaa862-e963-4b91-b362-9749d3014ab5 tempest-VolumesActionsTest-1847628510 None] Attach volume completed successfully. May 14 21:42:16.367928 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-45eaa862-e963-4b91-b362-9749d3014ab5 tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b'" released by "attach_volume" :: held 0.395s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:16.597946 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0e59fc96-270e-4fa9-a606-5b9b91c156a6 tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-detach_volume'" acquired by "detach_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:16.598523 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0e59fc96-270e-4fa9-a606-5b9b91c156a6 tempest-VolumesActionsTest-1847628510 None] Detaching volume c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b from instance 3ee751a8-3bfb-4b61-8117-6f8e05f6cd88. May 14 21:42:16.851231 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0e59fc96-270e-4fa9-a606-5b9b91c156a6 tempest-VolumesActionsTest-1847628510 None] Detach volume completed successfully. May 14 21:42:16.859003 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0e59fc96-270e-4fa9-a606-5b9b91c156a6 tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-detach_volume'" released by "detach_volume" :: held 0.261s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:23.089620 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-b4af849f-150c-43bf-82ab-3f737b725796 tempest-VolumesActionsTest-1847628510 None] Running cmd (subprocess): rbd export --pool volumes volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b /opt/stack/data/cinder/conversion/volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-b2f86cf4-5cfb-4c23-80ed-e7749edded99 --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:42:24.182400 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-b4af849f-150c-43bf-82ab-3f737b725796 tempest-VolumesActionsTest-1847628510 None] CMD "rbd export --pool volumes volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b /opt/stack/data/cinder/conversion/volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-b2f86cf4-5cfb-4c23-80ed-e7749edded99 --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.092s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:42:24.197479 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.image_utils [None req-b4af849f-150c-43bf-82ab-3f737b725796 tempest-VolumesActionsTest-1847628510 None] b2f86cf4-5cfb-4c23-80ed-e7749edded99 was raw, no need to convert to raw {{(pid=123870) upload_volume /opt/stack/cinder/cinder/image/image_utils.py:817}} May 14 21:42:26.789936 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:42:26.835058 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (67374968-f46f-4c50-9d1e-99c72a47b5cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:26.846126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9fc0b4ef-3fb6-4a64-88f0-89074b808bd5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:26.876969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9fc0b4ef-3fb6-4a64-88f0-89074b808bd5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:42:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1821591177',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=138a26ba-83df-4a43-8751-de93108899b2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0166571316634a7185b9768add179654',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:42:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:42:27Z,use_quota=True,user_id='a3f0269a87fb4a8fa8dbc1b84c134fe7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:26.885892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3818eca2-9c0f-4993-8b16-f9b723f87a11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:26.905542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3818eca2-9c0f-4993-8b16-f9b723f87a11) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:26.915453 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ecbd652-b9e2-4d30-ae35-603366a6b22d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:26.929860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ecbd652-b9e2-4d30-ae35-603366a6b22d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '138a26ba-83df-4a43-8751-de93108899b2', 'volume_name': 'volume-138a26ba-83df-4a43-8751-de93108899b2', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:26.929860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (38b88ecf-3d4a-43d1-bf2d-56039e46c28a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:26.941690 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (38b88ecf-3d4a-43d1-bf2d-56039e46c28a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:26.949259 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fd15307c-1a2e-4794-a95a-701aa4a5c9e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:26.950663 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Volume 138a26ba-83df-4a43-8751-de93108899b2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-138a26ba-83df-4a43-8751-de93108899b2', 'volume_size': 1} May 14 21:42:26.951258 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] creating volume 'volume-138a26ba-83df-4a43-8751-de93108899b2' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:42:26.952294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:27.731960 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:42:27.732345 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:42:27.736823 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fd15307c-1a2e-4794-a95a-701aa4a5c9e7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-138a26ba-83df-4a43-8751-de93108899b2', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:27.741769 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d81f2388-a6d4-4913-829d-bc6ae6fd9ec2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:27.870537 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Volume volume-138a26ba-83df-4a43-8751-de93108899b2 (138a26ba-83df-4a43-8751-de93108899b2): created successfully May 14 21:42:27.871408 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d81f2388-a6d4-4913-829d-bc6ae6fd9ec2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:27.909716 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (67374968-f46f-4c50-9d1e-99c72a47b5cc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:28.117114 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7284ccea-841b-48cd-a335-c8ec87343d35 tempest-VolumesActionsTest-1932666048 None] Created volume successfully. May 14 21:42:28.745458 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-958079f0-207e-4eb1-9ed4-c20f08c7260d tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2'" acquired by "attach_volume" :: waited 0.026s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:29.002207 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-958079f0-207e-4eb1-9ed4-c20f08c7260d tempest-VolumesActionsTest-1932666048 None] Attaching volume 138a26ba-83df-4a43-8751-de93108899b2 to instance 787596fc-6501-43f7-9502-4ccc956edc28 at mountpoint /dev/sdb on host None. May 14 21:42:29.100786 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-958079f0-207e-4eb1-9ed4-c20f08c7260d tempest-VolumesActionsTest-1932666048 None] Attach volume completed successfully. May 14 21:42:29.111604 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-958079f0-207e-4eb1-9ed4-c20f08c7260d tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2'" released by "attach_volume" :: held 0.365s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:29.999214 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d1759862-7082-4537-952a-0771e30a658f tempest-VolumesActionsTest-1629893462 None] Terminate volume connection completed successfully. May 14 21:42:30.045464 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d1759862-7082-4537-952a-0771e30a658f tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2-detach_volume'" acquired by "detach_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:30.057751 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d1759862-7082-4537-952a-0771e30a658f tempest-VolumesActionsTest-1629893462 None] Detaching volume 138a26ba-83df-4a43-8751-de93108899b2 from instance 787596fc-6501-43f7-9502-4ccc956edc28. May 14 21:42:30.313312 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d1759862-7082-4537-952a-0771e30a658f tempest-VolumesActionsTest-1629893462 None] Detach volume completed successfully. May 14 21:42:30.333112 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d1759862-7082-4537-952a-0771e30a658f tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2-detach_volume'" released by "detach_volume" :: held 0.288s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:30.614887 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-fb2764a6-b333-48b2-9951-65187bb3c9f2 tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2-detach_volume'" acquired by "detach_volume" :: waited 0.024s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:30.616892 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fb2764a6-b333-48b2-9951-65187bb3c9f2 tempest-VolumesActionsTest-1932666048 None] Volume detach called, but volume not attached. May 14 21:42:30.729016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-fb2764a6-b333-48b2-9951-65187bb3c9f2 tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2-detach_volume'" released by "detach_volume" :: held 0.115s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:38.879400 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:42:38.912758 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (be0fb788-3d63-4e67-8e09-25977dcae90c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:38.921306 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dec699ca-f75b-4c70-aebd-fa8761e15d59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:38.970364 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dec699ca-f75b-4c70-aebd-fa8761e15d59) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:42:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1887525845',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5aac8a13-a0c4-49a1-b51a-d6f48ae267f6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0166571316634a7185b9768add179654',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:42:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:42:39Z,use_quota=True,user_id='a3f0269a87fb4a8fa8dbc1b84c134fe7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:38.984679 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ee14e4e5-ca64-4bf6-a36f-309f68842236) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:38.991987 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ee14e4e5-ca64-4bf6-a36f-309f68842236) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:38.997652 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c3665efa-6feb-4f2f-a3b7-4c0d002e03c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:39.002442 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c3665efa-6feb-4f2f-a3b7-4c0d002e03c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5aac8a13-a0c4-49a1-b51a-d6f48ae267f6', 'volume_name': 'volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:39.008847 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (93e47e1c-1662-4bb9-9b44-7b7928ee4d34) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:39.013818 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (93e47e1c-1662-4bb9-9b44-7b7928ee4d34) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:39.019341 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (87286c26-3ecb-4bba-8e12-4813cae21139) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:39.020779 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Volume 5aac8a13-a0c4-49a1-b51a-d6f48ae267f6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6', 'volume_size': 1} May 14 21:42:39.021819 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] creating volume 'volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:42:39.022724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:39.750828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:42:39.751219 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:42:39.757704 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (87286c26-3ecb-4bba-8e12-4813cae21139) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:39.765601 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e1ceb5ab-a7b5-43ea-9c32-b6ce1ccda9d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:39.871737 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Volume volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6 (5aac8a13-a0c4-49a1-b51a-d6f48ae267f6): created successfully May 14 21:42:39.876893 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e1ceb5ab-a7b5-43ea-9c32-b6ce1ccda9d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:39.884378 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (be0fb788-3d63-4e67-8e09-25977dcae90c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:39.951009 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-13d914fe-bc1d-4368-a5b4-a59b7a64d0a3 tempest-VolumesActionsTest-1932666048 None] Created volume successfully. May 14 21:42:41.433100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:41.564940 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:41.784375 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:42:41.784832 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] Volume volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:42:41.888257 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] deleting rbd volume volume-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:42:45.918401 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] Created reservations ['dc73c44b-f6f5-4e4d-a4be-d80057ae6285', '6c1d1695-423b-4064-9580-72ea266db26e', '45eec672-b1ba-4ec9-a4df-90b2a7bc2297', 'e5825cd3-2176-4c73-847d-b8f6e37c646f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:42:46.222026 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:46.305751 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:46.388315 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:42:46.398899 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] Deleted volume successfully. May 14 21:42:46.486528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a74a945f-d3d1-4502-aaee-686b41866e24 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-5aac8a13-a0c4-49a1-b51a-d6f48ae267f6-delete_volume'" released by "delete_volume" :: held 5.053s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:47.958579 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:42:47.981468 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (2ebef9ee-6ab4-40ce-a0e2-00817d36e1fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:47.988850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e3e033e0-6f2f-4ecf-bc61-bcec65624389) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:48.006839 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e3e033e0-6f2f-4ecf-bc61-bcec65624389) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:42:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-994398311',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0166571316634a7185b9768add179654',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:42:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:42:48Z,use_quota=True,user_id='a3f0269a87fb4a8fa8dbc1b84c134fe7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:48.012414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a9ca6f2b-5a0d-4ddc-9d40-2252b9ee92e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:48.016792 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a9ca6f2b-5a0d-4ddc-9d40-2252b9ee92e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:48.024039 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23f98f7f-0fba-4c6d-aa6a-2c01fe5dffc7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:48.028639 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23f98f7f-0fba-4c6d-aa6a-2c01fe5dffc7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c', 'volume_name': 'volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:48.032826 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5428f40f-c93f-44e5-b497-1baa075ae9af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:48.037490 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5428f40f-c93f-44e5-b497-1baa075ae9af) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:48.043350 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a95f37c-2f6a-4d73-b4ab-835c532da88d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:48.044673 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Volume c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c', 'volume_size': 1} May 14 21:42:48.045167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] creating volume 'volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:42:48.046097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:48.629169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:42:48.629169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:42:48.629169 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a95f37c-2f6a-4d73-b4ab-835c532da88d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:48.637106 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a167475-899e-4c01-a754-d9c1a1b0c039) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:48.750077 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Volume volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c (c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c): created successfully May 14 21:42:48.762131 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a167475-899e-4c01-a754-d9c1a1b0c039) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:48.778058 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (2ebef9ee-6ab4-40ce-a0e2-00817d36e1fc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:48.863188 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-31164be9-120e-4e10-b637-9b85bb7ecf75 tempest-VolumesActionsTest-1932666048 None] Created volume successfully. May 14 21:42:49.424627 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:49.559514 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:49.801995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:42:49.809035 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] Volume volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:42:49.996753 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] deleting rbd volume volume-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:42:53.006588 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b4af849f-150c-43bf-82ab-3f737b725796 tempest-VolumesActionsTest-1847628510 None] Uploaded volume to glance image-id: b2f86cf4-5cfb-4c23-80ed-e7749edded99. {{(pid=123870) copy_volume_to_image /opt/stack/cinder/cinder/volume/manager.py:1747}} May 14 21:42:53.178268 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b4af849f-150c-43bf-82ab-3f737b725796 tempest-VolumesActionsTest-1847628510 None] Copy volume to image completed successfully. May 14 21:42:54.562046 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] Created reservations ['009e1ea2-7549-41dd-ade2-8c832968ff95', 'c407d342-289f-47fd-8686-73277ff96c74', 'd63b48d8-951a-4e61-9479-247dc919304b', '52a59003-8ca6-44e2-a109-b8724589b59f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:42:54.763013 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:54.804803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:54.843914 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:42:54.850422 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] Deleted volume successfully. May 14 21:42:54.860974 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f80637d8-d832-4a24-a247-19cbdaaac72c tempest-GroupSnapshotsTest-385817527 None] Group tempest-GroupSnapshotsTest-Group-1073229820: creating May 14 21:42:54.948216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0a7d622c-e75c-4ef8-adaf-f40e5f2aab60 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-c21bc2ff-5840-47cf-a43f-f7b3c7c6f43c-delete_volume'" released by "delete_volume" :: held 5.523s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:42:54.969970 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f80637d8-d832-4a24-a247-19cbdaaac72c tempest-GroupSnapshotsTest-385817527 None] Group tempest-GroupSnapshotsTest-Group-1073229820: created successfully May 14 21:42:54.978809 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f80637d8-d832-4a24-a247-19cbdaaac72c tempest-GroupSnapshotsTest-385817527 None] Create group completed successfully. May 14 21:42:55.377236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:55.488887 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:55.686452 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:42:55.687076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] Volume volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:42:55.796705 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] deleting rbd volume volume-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:42:55.889252 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:42:55.909528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (5b2f435c-d219-4890-a131-e9bd94af841b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:55.916669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (917479ba-07d2-40af-8a58-083830a24e85) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:55.932817 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (917479ba-07d2-40af-8a58-083830a24e85) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:42:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-641056263',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2ac64bd1-11fe-44f8-b773-624dd2853935,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0166571316634a7185b9768add179654',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:42:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:42:56Z,use_quota=True,user_id='a3f0269a87fb4a8fa8dbc1b84c134fe7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:55.938201 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7be6b7ee-fb01-4245-b276-65dfbd145cda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:55.943406 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7be6b7ee-fb01-4245-b276-65dfbd145cda) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:55.949040 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aa23cbaa-249e-4c1b-bd7a-aad1ea37ae7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:55.953144 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aa23cbaa-249e-4c1b-bd7a-aad1ea37ae7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2ac64bd1-11fe-44f8-b773-624dd2853935', 'volume_name': 'volume-2ac64bd1-11fe-44f8-b773-624dd2853935', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:55.958069 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (442671f9-407d-4c96-9a26-346d6445e284) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:55.962285 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (442671f9-407d-4c96-9a26-346d6445e284) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:55.967183 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0ba6d77b-416f-4c6d-a32d-e0cf53de0756) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:55.968306 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Volume 2ac64bd1-11fe-44f8-b773-624dd2853935: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2ac64bd1-11fe-44f8-b773-624dd2853935', 'volume_size': 1} May 14 21:42:55.968722 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] creating volume 'volume-2ac64bd1-11fe-44f8-b773-624dd2853935' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:42:55.969509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:56.786125 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:42:56.808198 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (cd64c224-cb86-4b35-9378-355fc6281394) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:56.814632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1806143-17d0-4476-a2ce-638f3afb9c99) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:56.829958 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1806143-17d0-4476-a2ce-638f3afb9c99) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:42:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1219369004',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=a4442e40-1cc6-4996-9622-78f512742cd7,host='ncdv-master-1@ceph#ceph',id=c7e3cec1-965d-43e1-8257-232173e4cae6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8a79a55f6b2f46ae89036ef9bb98639c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:42:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:42:57Z,use_quota=True,user_id='16ffd497e45e4683971ebee0eaebb975',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(ad08780c-700c-4621-ac1d-d5fb19203115),volume_type_id=ad08780c-700c-4621-ac1d-d5fb19203115)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:56.836031 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ce22d446-3d98-411b-86d8-2115f816797e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:56.840228 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ce22d446-3d98-411b-86d8-2115f816797e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:56.845348 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (25bc358d-13d6-406b-8cba-f406153f4c2b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:56.850006 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (25bc358d-13d6-406b-8cba-f406153f4c2b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c7e3cec1-965d-43e1-8257-232173e4cae6', 'volume_name': 'volume-c7e3cec1-965d-43e1-8257-232173e4cae6', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:56.854948 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d641db5-94f5-4f02-afa9-fcbb805b71fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:56.858901 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d641db5-94f5-4f02-afa9-fcbb805b71fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:56.863699 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e028f7e8-c227-46c2-891a-4cb56b2c1420) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:56.864866 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Volume c7e3cec1-965d-43e1-8257-232173e4cae6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c7e3cec1-965d-43e1-8257-232173e4cae6', 'volume_size': 1} May 14 21:42:56.865262 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] creating volume 'volume-c7e3cec1-965d-43e1-8257-232173e4cae6' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:42:56.866011 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:57.044307 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:42:57.044656 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:42:57.048533 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0ba6d77b-416f-4c6d-a32d-e0cf53de0756) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2ac64bd1-11fe-44f8-b773-624dd2853935', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:57.053863 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf5f1c97-b8d9-4da8-83e4-0d8304bca89f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:57.127317 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Volume volume-2ac64bd1-11fe-44f8-b773-624dd2853935 (2ac64bd1-11fe-44f8-b773-624dd2853935): created successfully May 14 21:42:57.131585 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf5f1c97-b8d9-4da8-83e4-0d8304bca89f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:57.137145 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (5b2f435c-d219-4890-a131-e9bd94af841b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:57.200457 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bcffef3e-39fc-4ea0-bba8-fe94b89204d0 tempest-VolumesActionsTest-1932666048 None] Created volume successfully. May 14 21:42:57.617669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:42:57.618055 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:42:57.622269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e028f7e8-c227-46c2-891a-4cb56b2c1420) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c7e3cec1-965d-43e1-8257-232173e4cae6', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:57.628337 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1fc83b8a-c620-4d2c-8854-d14a99430f17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:42:57.738149 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Volume volume-c7e3cec1-965d-43e1-8257-232173e4cae6 (c7e3cec1-965d-43e1-8257-232173e4cae6): created successfully May 14 21:42:57.742287 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1fc83b8a-c620-4d2c-8854-d14a99430f17) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:42:57.747952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (cd64c224-cb86-4b35-9378-355fc6281394) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:42:57.859249 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8db0ef94-d8e2-4b4c-a540-528fe0791fd5 tempest-GroupSnapshotsTest-385817527 None] Created volume successfully. May 14 21:42:58.057175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-2ac64bd1-11fe-44f8-b773-624dd2853935-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:42:58.163958 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:42:58.350190 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:42:58.350528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] Volume volume-2ac64bd1-11fe-44f8-b773-624dd2853935 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:42:58.566303 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] deleting rbd volume volume-2ac64bd1-11fe-44f8-b773-624dd2853935 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:42:59.378326 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-63f113a5-08e3-4d85-8480-72cc0119a86e tempest-GroupSnapshotsTest-385817527 None] GroupSnapshot 6eb70917-f64c-4686-882b-33dd2625ce31: creating. May 14 21:42:59.397685 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-63f113a5-08e3-4d85-8480-72cc0119a86e tempest-GroupSnapshotsTest-385817527 None] Group snapshot 6eb70917-f64c-4686-882b-33dd2625ce31: creating. {{(pid=123870) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4078}} May 14 21:42:59.416428 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-63f113a5-08e3-4d85-8480-72cc0119a86e tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:02.007405 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] Created reservations ['0fde7d6e-2541-457c-bf69-4eaa86344605', '601992bf-437e-42b2-acf8-86df7c507f54', '1b4aff2a-5c8b-4dea-9525-e9215ad03a17', 'bf8ffec1-4989-4366-ab7a-241ea74c7135'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:02.185080 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:02.217528 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:02.242777 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:02.248176 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] Deleted volume successfully. May 14 21:43:02.314740 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bbf7b99e-a6f5-4f24-985e-2f5b06c0204a tempest-VolumesActionsTest-1847628510 None] Lock "b'cinder-c029fd91-fe3f-4f32-b1f8-6f65bb4ab00b-delete_volume'" released by "delete_volume" :: held 6.932s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:02.710122 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-63f113a5-08e3-4d85-8480-72cc0119a86e tempest-GroupSnapshotsTest-385817527 None] group_snapshot 6eb70917-f64c-4686-882b-33dd2625ce31: created successfully May 14 21:43:03.727733 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:43:03.728508 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:03.749937 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:03.773709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:03.900130 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] Created reservations ['00226dd3-526c-41f7-865b-de303707a4cc', '2a1f4e91-f36e-4ad0-93a0-05def87f204c', '33ce165d-5643-45e4-8563-fbdc74e03bfd', '0485d81c-deb0-4617-869b-de90ef30468d'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:04.117059 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:04.140381 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:04.168635 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:04.173236 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] Deleted volume successfully. May 14 21:43:04.228622 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-56c45cad-02ec-4f21-98ea-f84141d39ad3 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-2ac64bd1-11fe-44f8-b773-624dd2853935-delete_volume'" released by "delete_volume" :: held 6.171s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:04.588165 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d16969bf-f15a-461d-9522-e4ad0e35d7ec tempest-GroupSnapshotsTest-385817527 None] cloning volumes/volume-c7e3cec1-965d-43e1-8257-232173e4cae6@snapshot-7e99188d-6af7-477b-99b6-8242a11da058 to volume-c9480c6b-2618-4acc-a134-8613730eab18 {{(pid=123870) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}} May 14 21:43:04.588987 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d16969bf-f15a-461d-9522-e4ad0e35d7ec tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:04.636060 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d16969bf-f15a-461d-9522-e4ad0e35d7ec tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:05.254235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:05.275277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (88b1bc87-402d-4bdf-a5fe-837f0af0eae8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:05.281632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d22986ce-0638-4589-8337-b6ec2f934cf3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:05.305601 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d22986ce-0638-4589-8337-b6ec2f934cf3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-717258178',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=697a0270-6623-4cd5-802e-f2575bfac3e4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0166571316634a7185b9768add179654',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:05Z,use_quota=True,user_id='a3f0269a87fb4a8fa8dbc1b84c134fe7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:05.311122 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (10dfc1be-187a-4244-92b1-0c619478f704) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:05.315328 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (10dfc1be-187a-4244-92b1-0c619478f704) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:05.321371 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bbbd6718-e1da-46f1-8499-7dc5534fde66) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:05.325436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bbbd6718-e1da-46f1-8499-7dc5534fde66) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '697a0270-6623-4cd5-802e-f2575bfac3e4', 'volume_name': 'volume-697a0270-6623-4cd5-802e-f2575bfac3e4', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:05.330433 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (56bf6134-18ec-445f-9033-7969422d5ff7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:05.335161 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (56bf6134-18ec-445f-9033-7969422d5ff7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:05.340014 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0761ee98-0822-406b-8500-b1a23be7ae29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:05.341111 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Volume 697a0270-6623-4cd5-802e-f2575bfac3e4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-697a0270-6623-4cd5-802e-f2575bfac3e4', 'volume_size': 1} May 14 21:43:05.341512 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] creating volume 'volume-697a0270-6623-4cd5-802e-f2575bfac3e4' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:05.342174 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:05.948577 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d16969bf-f15a-461d-9522-e4ad0e35d7ec tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:05.948951 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d16969bf-f15a-461d-9522-e4ad0e35d7ec tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:06.263914 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:06.264451 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:06.268512 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0761ee98-0822-406b-8500-b1a23be7ae29) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-697a0270-6623-4cd5-802e-f2575bfac3e4', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:06.273622 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9852ad95-dcf5-456f-8994-c19e5553918d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:06.356316 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Volume volume-697a0270-6623-4cd5-802e-f2575bfac3e4 (697a0270-6623-4cd5-802e-f2575bfac3e4): created successfully May 14 21:43:06.362664 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9852ad95-dcf5-456f-8994-c19e5553918d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:06.368626 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (88b1bc87-402d-4bdf-a5fe-837f0af0eae8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:06.486395 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8a59d5d5-eafa-4e73-a4f3-2ad45a0a01bb tempest-VolumesActionsTest-1932666048 None] Created volume successfully. May 14 21:43:06.535951 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:06.555835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Flow 'volume_create_manager' (a5eeaf0a-1327-49c6-8b37-60e56b09dd8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:06.562482 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6557783-5024-429c-a53c-020bc7b492d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:06.582850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6557783-5024-429c-a53c-020bc7b492d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-807472151',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f2219050-2175-4ef5-b7e6-5497a584ba80,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8526cad5fef54772a988df44138c90fe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:06Z,use_quota=True,user_id='80f409708620474aa128bf8412fa54be',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:06.588947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d525744c-a29a-4bdc-b23c-e4a665e520f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:06.593460 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d525744c-a29a-4bdc-b23c-e4a665e520f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:06.598572 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c82fb9b5-5146-40b2-abca-cd56a6248092) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:06.603283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c82fb9b5-5146-40b2-abca-cd56a6248092) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f2219050-2175-4ef5-b7e6-5497a584ba80', 'volume_name': 'volume-f2219050-2175-4ef5-b7e6-5497a584ba80', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:06.608218 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1ec78d60-47c3-44bb-8232-ef20ea1fe894) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:06.612141 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1ec78d60-47c3-44bb-8232-ef20ea1fe894) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:06.617052 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5599fba9-49fa-40eb-9566-2f6f5a165025) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:06.618137 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Volume f2219050-2175-4ef5-b7e6-5497a584ba80: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f2219050-2175-4ef5-b7e6-5497a584ba80', 'volume_size': 1} May 14 21:43:06.618533 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] creating volume 'volume-f2219050-2175-4ef5-b7e6-5497a584ba80' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:06.619206 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:06.648668 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d16969bf-f15a-461d-9522-e4ad0e35d7ec tempest-GroupSnapshotsTest-385817527 None] Create group from source-snapshot-6eb70917-f64c-4686-882b-33dd2625ce31 completed successfully. May 14 21:43:07.314226 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:07.321595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:07.321595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5599fba9-49fa-40eb-9566-2f6f5a165025) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f2219050-2175-4ef5-b7e6-5497a584ba80', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:07.332990 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (de6e8107-b901-4b44-a192-ce3201eecf6a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:07.425542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:07.590628 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:07.671299 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] deleting rbd volume volume-c9480c6b-2618-4acc-a134-8613730eab18 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:07.679460 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Volume volume-f2219050-2175-4ef5-b7e6-5497a584ba80 (f2219050-2175-4ef5-b7e6-5497a584ba80): created successfully May 14 21:43:07.683185 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (de6e8107-b901-4b44-a192-ce3201eecf6a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:07.689340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Flow 'volume_create_manager' (a5eeaf0a-1327-49c6-8b37-60e56b09dd8a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:07.703081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-697a0270-6623-4cd5-802e-f2575bfac3e4-delete_volume'" acquired by "delete_volume" :: waited 0.027s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:07.800343 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7934bfc3-f8aa-4889-9907-f25c97e9beb1 tempest-VolumesBackupsTest-1778047115 None] Created volume successfully. May 14 21:43:07.835400 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:08.150209 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:08.150666 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] Volume volume-697a0270-6623-4cd5-802e-f2575bfac3e4 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:08.430616 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] deleting rbd volume volume-697a0270-6623-4cd5-802e-f2575bfac3e4 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:14.978409 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] Created reservations ['38523939-671c-48b5-be9e-411e98701ec3', '651ff205-f54e-4b45-9d08-7efefea340dc', '385d66e5-7446-498f-bace-3eb24125fe5e', '1d96e046-9ffe-4b67-b16c-9b0a089f860c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:15.199070 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['fa8a31e4-98d2-4ed9-9896-89f67dd38ae8'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:15.226268 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1185674682 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1185674682, it is now deprecated. Please use the default quota class for default quota. May 14 21:43:15.227056 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1185674682 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1185674682, it is now deprecated. Please use the default quota class for default quota. May 14 21:43:15.290868 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:15.330813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:15.371456 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:15.376433 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] Deleted volume successfully. May 14 21:43:15.459652 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ddb5bac0-dfe9-468a-94f8-3cca5c881039 tempest-VolumesActionsTest-1629893462 None] Lock "b'cinder-697a0270-6623-4cd5-802e-f2575bfac3e4-delete_volume'" released by "delete_volume" :: held 7.751s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:15.476316 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['9fcf8952-ceaa-4763-8912-1283e8ce3a80', '1b3c86a0-ed32-444d-8b62-40a75eec6954', '7cf4a8c2-4785-4b98-afaa-d816697bebc3', 'a0e04b21-e1a0-4917-842e-1522b7ff4d01'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:15.894530 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:15.929521 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:15.967304 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:15.971593 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a4d65fe1-e4f2-4dee-95d6-05af7a2767f1 tempest-GroupSnapshotsTest-385817527 None] Delete group completed successfully. May 14 21:43:16.333796 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5fd5f5dc-5ffd-4569-a61c-bc9f8006d27d tempest-GroupSnapshotsTest-385817527 None] group_snapshot 6eb70917-f64c-4686-882b-33dd2625ce31: deleting May 14 21:43:16.354755 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5fd5f5dc-5ffd-4569-a61c-bc9f8006d27d tempest-GroupSnapshotsTest-385817527 None] group_snapshot 6eb70917-f64c-4686-882b-33dd2625ce31: deleting {{(pid=123870) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4253}} May 14 21:43:16.376121 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5fd5f5dc-5ffd-4569-a61c-bc9f8006d27d tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:17.105224 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:17.125997 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (eeca2026-561d-45f2-9b54-76f39e4dfd1e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:17.133200 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (92a95d5a-db59-4bf9-a8e9-8b054a73ed02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:17.148420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (92a95d5a-db59-4bf9-a8e9-8b054a73ed02) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-138401936',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ffffed0c-05cf-4896-97ef-140e531b7ac7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0166571316634a7185b9768add179654',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:17Z,use_quota=True,user_id='a3f0269a87fb4a8fa8dbc1b84c134fe7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:17.153808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (20f0be8a-0433-4ffe-b4bb-8bd1579e79fd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:17.157938 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (20f0be8a-0433-4ffe-b4bb-8bd1579e79fd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:17.163083 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9e4a5a5-37ef-469c-b93f-1193ae2734c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:17.167115 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9e4a5a5-37ef-469c-b93f-1193ae2734c9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ffffed0c-05cf-4896-97ef-140e531b7ac7', 'volume_name': 'volume-ffffed0c-05cf-4896-97ef-140e531b7ac7', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:17.172131 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (571f78ca-8eac-4a03-af54-5d10cec9113a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:17.176019 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (571f78ca-8eac-4a03-af54-5d10cec9113a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:17.183839 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (15e4d574-86d5-4789-b176-f7c1ef811d84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:17.185272 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Volume ffffed0c-05cf-4896-97ef-140e531b7ac7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ffffed0c-05cf-4896-97ef-140e531b7ac7', 'volume_size': 1} May 14 21:43:17.185777 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] creating volume 'volume-ffffed0c-05cf-4896-97ef-140e531b7ac7' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:17.186789 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:17.884283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:17.885597 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:17.889652 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (15e4d574-86d5-4789-b176-f7c1ef811d84) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ffffed0c-05cf-4896-97ef-140e531b7ac7', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:17.900065 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89b9970e-f381-4b8f-aae4-cef15eb6546c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:17.972529 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Volume volume-ffffed0c-05cf-4896-97ef-140e531b7ac7 (ffffed0c-05cf-4896-97ef-140e531b7ac7): created successfully May 14 21:43:17.976469 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89b9970e-f381-4b8f-aae4-cef15eb6546c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:17.982338 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Flow 'volume_create_manager' (eeca2026-561d-45f2-9b54-76f39e4dfd1e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:18.070301 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4e4f98f7-7fb1-4c36-af06-06b295ef1e55 tempest-VolumesActionsTest-1932666048 None] Created volume successfully. May 14 21:43:18.873361 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-5fd5f5dc-5ffd-4569-a61c-bc9f8006d27d tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1185674682 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1185674682, it is now deprecated. Please use the default quota class for default quota. May 14 21:43:18.960173 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-5fd5f5dc-5ffd-4569-a61c-bc9f8006d27d tempest-GroupSnapshotsTest-385817527 None] Created reservations ['c2ab294d-1d4e-4db2-b723-cbe15b1d94cf', 'd1c6c408-5d23-4ff3-a2ed-be68c859a4d4', 'b8e39eb7-7df9-4f2a-aa47-7e7f1db70643', 'd189db71-aa6f-4f25-a6b0-a3362af2e45b'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:19.218243 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5fd5f5dc-5ffd-4569-a61c-bc9f8006d27d tempest-GroupSnapshotsTest-385817527 None] group_snapshot 6eb70917-f64c-4686-882b-33dd2625ce31: deleted successfully May 14 21:43:19.758828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-ffffed0c-05cf-4896-97ef-140e531b7ac7-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:19.894291 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:20.060118 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:20.060750 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] Volume volume-ffffed0c-05cf-4896-97ef-140e531b7ac7 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:20.138829 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] deleting rbd volume volume-ffffed0c-05cf-4896-97ef-140e531b7ac7 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:20.169881 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:20.394938 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:20.395468 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] Volume volume-c7e3cec1-965d-43e1-8257-232173e4cae6 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:20.596626 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] deleting rbd volume volume-c7e3cec1-965d-43e1-8257-232173e4cae6 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:23.685833 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-975b0ec5-dd17-47dc-91c1-09e929eebd92 req-d1bd41bd-e763-48e7-93f0-c5af7bffba62 tempest-VolumesBackupsTest-1778047115 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:43:24.497436 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-975b0ec5-dd17-47dc-91c1-09e929eebd92 req-d1bd41bd-e763-48e7-93f0-c5af7bffba62 tempest-VolumesBackupsTest-1778047115 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.811s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:43:24.498247 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-975b0ec5-dd17-47dc-91c1-09e929eebd92 req-d1bd41bd-e763-48e7-93f0-c5af7bffba62 tempest-VolumesBackupsTest-1778047115 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f2219050-2175-4ef5-b7e6-5497a584ba80', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'f2219050-2175-4ef5-b7e6-5497a584ba80', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:43:24.617683 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-975b0ec5-dd17-47dc-91c1-09e929eebd92 req-d1bd41bd-e763-48e7-93f0-c5af7bffba62 tempest-VolumesBackupsTest-1778047115 None] Connection info returned from driver {'name': 'volumes/volume-f2219050-2175-4ef5-b7e6-5497a584ba80', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f2219050-2175-4ef5-b7e6-5497a584ba80', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '34687cf4-b7bc-4316-ace6-e66ff7214212'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:43:24.766756 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-975b0ec5-dd17-47dc-91c1-09e929eebd92 req-d1bd41bd-e763-48e7-93f0-c5af7bffba62 tempest-VolumesBackupsTest-1778047115 None] attachment_update completed successfully. May 14 21:43:26.922002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] Created reservations ['95c1d348-a31f-4cb1-97e2-1cf94860ae18', '32c292c7-b387-46ac-97d1-1d38413cb064', '182a9b5b-67fe-41d2-966f-7484e6cd46f3', '6867e9b3-fea4-404a-8d76-ef2df72af84e'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:27.158539 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:27.257555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:27.334559 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:27.363334 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] Deleted volume successfully. May 14 21:43:27.532360 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5e3c0e9-045c-4ef6-9222-67a71d069326 tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-ffffed0c-05cf-4896-97ef-140e531b7ac7-delete_volume'" released by "delete_volume" :: held 7.773s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:27.648500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:27.664186 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['cd2ae89f-06d7-4bde-807c-c4c6bd559405'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:27.700190 ncdv-master-1 cinder-volume[123870]: WARNING cinder.db.sqlalchemy.api [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] Reservation would make usage less than 0 for the following resources, so on commit they will be limited to prevent going below 0: ['gigabytes', 'gigabytes_tempest-GroupSnapshotsTest-volume-type-1185674682'] May 14 21:43:27.781344 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:27.966204 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:27.982751 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] Volume volume-138a26ba-83df-4a43-8751-de93108899b2 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:28.190781 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] deleting rbd volume volume-138a26ba-83df-4a43-8751-de93108899b2 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:28.195952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['e47ab0e6-736c-44b8-8948-14c8df2f23ef', '093bbd99-d0a1-4bd3-bb48-7b2d760b579c', '5749d510-d74f-4182-84ee-5f217784d344', 'da2e4778-33ff-40af-8402-5f6bc07020e7'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:28.720620 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:28.806643 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:28.870642 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:28.875079 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-45e3e581-374f-4f05-8ab4-c366cad8ae82 tempest-GroupSnapshotsTest-385817527 None] Delete group completed successfully. May 14 21:43:29.024300 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:43:28Z,data_timestamp=2022-05-14T21:43:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-541977381',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=0b889b5f-0acf-4c35-9afa-d6d9e482fb22,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='8526cad5fef54772a988df44138c90fe',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='80f409708620474aa128bf8412fa54be',volume_id=f2219050-2175-4ef5-b7e6-5497a584ba80) May 14 21:43:29.025336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] continue_backup in rpcapi backup_id 0b889b5f-0acf-4c35-9afa-d6d9e482fb22 {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:43:29.291760 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:43:30.098446 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.806s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:43:30.100330 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f2219050-2175-4ef5-b7e6-5497a584ba80', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'f2219050-2175-4ef5-b7e6-5497a584ba80', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:43:30.118369 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] Initialize volume connection completed successfully. May 14 21:43:30.544271 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-75fa4880-74e1-4e6c-bf6a-bca49beffbdc tempest-GroupSnapshotsTest-385817527 None] Group tempest-GroupSnapshotsTest-Group-187810179: creating May 14 21:43:30.619574 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-75fa4880-74e1-4e6c-bf6a-bca49beffbdc tempest-GroupSnapshotsTest-385817527 None] Group tempest-GroupSnapshotsTest-Group-187810179: created successfully May 14 21:43:30.631548 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-75fa4880-74e1-4e6c-bf6a-bca49beffbdc tempest-GroupSnapshotsTest-385817527 None] Create group completed successfully. May 14 21:43:32.355957 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:32.376171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (3de40480-52df-4330-9451-5cc0e6e22da6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:32.383713 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ef624ce9-e233-4b39-9b3d-d9d9d233527f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:32.400962 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ef624ce9-e233-4b39-9b3d-d9d9d233527f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1981992096',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=bcc5cc07-bfc2-46ba-aaa0-40452c9a5f6c,host='ncdv-master-1@ceph#ceph',id=3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8a79a55f6b2f46ae89036ef9bb98639c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:32Z,use_quota=True,user_id='16ffd497e45e4683971ebee0eaebb975',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(ec61e42d-7dd4-4d1b-ae1a-5b2a0b348cf4),volume_type_id=ec61e42d-7dd4-4d1b-ae1a-5b2a0b348cf4)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:32.406269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cc93de2b-fe05-4f59-8375-5a9358d1dec8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:32.410336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cc93de2b-fe05-4f59-8375-5a9358d1dec8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:32.415554 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c11e583d-57a4-4162-a15f-06b535df13ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:32.419439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c11e583d-57a4-4162-a15f-06b535df13ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e', 'volume_name': 'volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:32.424296 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (58537789-2bc7-4a70-9a3e-39763c95ab7c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:32.428393 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (58537789-2bc7-4a70-9a3e-39763c95ab7c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:32.433074 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (212834d5-b445-4c6e-b583-8e784e91486a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:32.434230 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Volume 3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e', 'volume_size': 1} May 14 21:43:32.434644 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] creating volume 'volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:32.435897 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:32.583187 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] Created reservations ['fd5097bc-5066-478d-b245-1865c296dd51', '33086673-a4fc-46d5-9da7-5f45837f9423', 'e36bf9ac-d5c0-4a94-af63-2fe177ca387a', '79668b85-1edb-4f4e-80ac-5bfef75052da'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:32.801697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:32.828053 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:32.855651 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:32.863813 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] Deleted volume successfully. May 14 21:43:32.937621 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ac8c8c76-02e9-4e0c-924e-f7a6b6851124 tempest-VolumesActionsTest-1932666048 None] Lock "b'cinder-138a26ba-83df-4a43-8751-de93108899b2-delete_volume'" released by "delete_volume" :: held 5.289s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:33.237548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:33.237962 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:33.241897 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (212834d5-b445-4c6e-b583-8e784e91486a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:33.246574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0f2bd914-7133-4797-b454-6ac08e65cd14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:33.315225 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Volume volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e (3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e): created successfully May 14 21:43:33.319498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0f2bd914-7133-4797-b454-6ac08e65cd14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:33.328957 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (3de40480-52df-4330-9451-5cc0e6e22da6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:33.407264 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8ab6b1c0-4872-4840-8dd0-8ed34049d5fe tempest-GroupSnapshotsTest-385817527 None] Created volume successfully. May 14 21:43:35.186999 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:35.208026 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (4e16b93b-c60b-4faa-9b44-0281bd4df9b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:35.214596 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2ee09f02-d3ba-44e0-b760-ff229fc44aac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:35.234448 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2ee09f02-d3ba-44e0-b760-ff229fc44aac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1270744902',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=bcc5cc07-bfc2-46ba-aaa0-40452c9a5f6c,host='ncdv-master-1@ceph#ceph',id=17b5e099-759d-4436-a1c5-fea8e6ad081d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8a79a55f6b2f46ae89036ef9bb98639c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:35Z,use_quota=True,user_id='16ffd497e45e4683971ebee0eaebb975',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(ec61e42d-7dd4-4d1b-ae1a-5b2a0b348cf4),volume_type_id=ec61e42d-7dd4-4d1b-ae1a-5b2a0b348cf4)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:35.236660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03ae5193-17c7-4b11-9c59-f0bb4a6423b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:35.242146 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03ae5193-17c7-4b11-9c59-f0bb4a6423b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:35.251686 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3513c01d-24f9-436c-b23d-2f4ab8dc20cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:35.252343 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3513c01d-24f9-436c-b23d-2f4ab8dc20cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '17b5e099-759d-4436-a1c5-fea8e6ad081d', 'volume_name': 'volume-17b5e099-759d-4436-a1c5-fea8e6ad081d', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:35.258219 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (976413ff-4e61-4ca9-9468-8b5e2f7778df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:35.262240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (976413ff-4e61-4ca9-9468-8b5e2f7778df) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:35.267075 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b38d9576-aa22-4894-8618-0a999e6bf03d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:35.268424 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Volume 17b5e099-759d-4436-a1c5-fea8e6ad081d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-17b5e099-759d-4436-a1c5-fea8e6ad081d', 'volume_size': 1} May 14 21:43:35.268869 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] creating volume 'volume-17b5e099-759d-4436-a1c5-fea8e6ad081d' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:35.274545 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:36.128752 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] Terminate volume connection completed successfully. May 14 21:43:36.145166 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-54cd9ff4-e261-4944-89e5-ce1b9ea6714e tempest-VolumesBackupsTest-1778047115 None] Remove volume export completed successfully. May 14 21:43:36.222189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:36.226298 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:36.240574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b38d9576-aa22-4894-8618-0a999e6bf03d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-17b5e099-759d-4436-a1c5-fea8e6ad081d', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:36.260076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6551e523-b50d-427b-8de8-48421ac98827) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:36.319895 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Volume volume-17b5e099-759d-4436-a1c5-fea8e6ad081d (17b5e099-759d-4436-a1c5-fea8e6ad081d): created successfully May 14 21:43:36.324077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6551e523-b50d-427b-8de8-48421ac98827) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:36.329757 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (4e16b93b-c60b-4faa-9b44-0281bd4df9b9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:36.383934 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d329461f-7272-4afd-866d-01b0bcba6b88 tempest-GroupSnapshotsTest-385817527 None] Created volume successfully. May 14 21:43:37.387463 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2ac13e95-395f-49bb-931b-325dcaf9c24e tempest-GroupSnapshotsTest-385817527 None] Update group completed successfully. May 14 21:43:38.658047 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6a12db92-e199-41b7-ba1b-123bf2a90c7f tempest-GroupSnapshotsTest-385817527 None] Update group completed successfully. May 14 21:43:39.373559 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:39.392780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Flow 'volume_create_manager' (ac0e26d9-5ecf-4f34-a5b0-334c4847edad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:39.399858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69d0b95d-83d8-4818-893c-2e1f0af18ed9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:39.415269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69d0b95d-83d8-4818-893c-2e1f0af18ed9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-1743839407',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f87d52f8-7a10-48c9-a4e4-8edd88e288be,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fae4cdc60e2f46bf8189dce15ab62978',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:39Z,use_quota=True,user_id='33e0486e4f744523949c71263aca0893',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:39.420790 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ae9cc96-cb63-4ea7-9d8c-3b20d0e7a7b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:39.424985 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ae9cc96-cb63-4ea7-9d8c-3b20d0e7a7b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:39.430849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (35523714-864b-41e0-bb76-cbb09bc66219) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:39.434925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (35523714-864b-41e0-bb76-cbb09bc66219) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f87d52f8-7a10-48c9-a4e4-8edd88e288be', 'volume_name': 'volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:39.439906 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c94bbdfe-2e57-486d-ad2d-b0c6bc08db20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:39.443858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c94bbdfe-2e57-486d-ad2d-b0c6bc08db20) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:39.449398 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b3aea9e2-9eed-42bb-bd3c-9795b3a49324) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:39.450521 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Volume f87d52f8-7a10-48c9-a4e4-8edd88e288be: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be', 'volume_size': 1} May 14 21:43:39.450898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] creating volume 'volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:39.451601 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:39.856408 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ff6cbfff-7386-4a9d-9292-eeda6681f897 tempest-GroupSnapshotsTest-385817527 None] Update group completed successfully. May 14 21:43:40.431962 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:40.432487 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:40.436277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b3aea9e2-9eed-42bb-bd3c-9795b3a49324) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:40.441292 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a22733a5-44b8-42ee-a578-5e6e5eb0ca16) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:40.526398 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Volume volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be (f87d52f8-7a10-48c9-a4e4-8edd88e288be): created successfully May 14 21:43:40.530912 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a22733a5-44b8-42ee-a578-5e6e5eb0ca16) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:40.535805 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Flow 'volume_create_manager' (ac0e26d9-5ecf-4f34-a5b0-334c4847edad) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:40.594701 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-151f1b6a-0011-4981-ae46-2815cdfc8b0a tempest-VolumesListAdminTestJSON-1202706947 None] Created volume successfully. May 14 21:43:41.013129 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5d49160e-e88e-4548-8699-fc4f1da2a278 tempest-GroupSnapshotsTest-385817527 None] Update group completed successfully. May 14 21:43:42.093879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:42.113853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Flow 'volume_create_manager' (074fc6d2-6967-4f3e-87c9-8bde21bfcc7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:42.120967 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (af8e21eb-0a18-472a-88c0-d8cda0d1299c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:42.136366 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (af8e21eb-0a18-472a-88c0-d8cda0d1299c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-446342402',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b00305bf-f72d-407d-8936-6f45ba59ef2c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fae4cdc60e2f46bf8189dce15ab62978',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:42Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:42Z,use_quota=True,user_id='33e0486e4f744523949c71263aca0893',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:42.141669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fbb868ff-d315-4b96-bd17-0c27857a86ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:42.145766 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fbb868ff-d315-4b96-bd17-0c27857a86ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:42.151098 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2c22618e-3b3e-4099-8924-23b1c3ca8ec3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:42.155215 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2c22618e-3b3e-4099-8924-23b1c3ca8ec3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b00305bf-f72d-407d-8936-6f45ba59ef2c', 'volume_name': 'volume-b00305bf-f72d-407d-8936-6f45ba59ef2c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:42.160099 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2a152a86-6411-4d93-8f40-c6c1fef698f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:42.164447 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2a152a86-6411-4d93-8f40-c6c1fef698f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:42.169310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (39f8b37c-f76a-4277-9562-1f1126014533) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:42.170381 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Volume b00305bf-f72d-407d-8936-6f45ba59ef2c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b00305bf-f72d-407d-8936-6f45ba59ef2c', 'volume_size': 1} May 14 21:43:42.170764 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] creating volume 'volume-b00305bf-f72d-407d-8936-6f45ba59ef2c' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:42.171426 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:42.527526 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-42852489-8765-4ab4-bd84-813c67b69a1c tempest-GroupSnapshotsTest-385817527 None] GroupSnapshot 92f2c388-9126-4e29-8f4b-cc9c3066532e: creating. May 14 21:43:42.543425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-42852489-8765-4ab4-bd84-813c67b69a1c tempest-GroupSnapshotsTest-385817527 None] Group snapshot 92f2c388-9126-4e29-8f4b-cc9c3066532e: creating. {{(pid=123870) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4078}} May 14 21:43:42.559579 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42852489-8765-4ab4-bd84-813c67b69a1c tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:43.512325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:43.513060 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:43.516611 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (39f8b37c-f76a-4277-9562-1f1126014533) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b00305bf-f72d-407d-8936-6f45ba59ef2c', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:43.521308 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aadb8b1b-c213-494e-a6f3-dfb133f45351) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:43.616909 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Volume volume-b00305bf-f72d-407d-8936-6f45ba59ef2c (b00305bf-f72d-407d-8936-6f45ba59ef2c): created successfully May 14 21:43:43.621412 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aadb8b1b-c213-494e-a6f3-dfb133f45351) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:43.626988 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Flow 'volume_create_manager' (074fc6d2-6967-4f3e-87c9-8bde21bfcc7a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:43.732027 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-445c8002-ed67-4a05-962d-69c037008bcb tempest-VolumesListAdminTestJSON-1202706947 None] Created volume successfully. May 14 21:43:44.704999 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:44.726126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Flow 'volume_create_manager' (3a3f626c-6cf7-4c65-b5bb-9b4cc583cd9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:44.733696 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (821c9021-7c23-43b1-b621-3a7ef85f81d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:44.749037 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (821c9021-7c23-43b1-b621-3a7ef85f81d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-2053396206',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fae4cdc60e2f46bf8189dce15ab62978',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:45Z,use_quota=True,user_id='33e0486e4f744523949c71263aca0893',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:44.754393 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (794f89f5-2757-4a9c-9341-c39d9342f679) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:44.758424 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (794f89f5-2757-4a9c-9341-c39d9342f679) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:44.764202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6ab98580-de2e-45d4-b67a-a83e6dab3cb6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:44.768211 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6ab98580-de2e-45d4-b67a-a83e6dab3cb6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed', 'volume_name': 'volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:44.773363 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (55734c5b-bce5-4b9b-a924-eb69708fd65a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:44.779315 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (55734c5b-bce5-4b9b-a924-eb69708fd65a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:44.782947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (415c26dc-585b-4480-b734-4b05991a9de9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:44.784166 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Volume c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed', 'volume_size': 1} May 14 21:43:44.784418 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] creating volume 'volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:44.785136 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:45.239084 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-42852489-8765-4ab4-bd84-813c67b69a1c tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:45.749931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:45.750619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:45.754462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (415c26dc-585b-4480-b734-4b05991a9de9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:45.759159 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d5408aa0-0a6b-429d-8096-75b1d9c535e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:45.831497 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Volume volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed (c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed): created successfully May 14 21:43:45.836654 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d5408aa0-0a6b-429d-8096-75b1d9c535e7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:45.842636 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Flow 'volume_create_manager' (3a3f626c-6cf7-4c65-b5bb-9b4cc583cd9e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:45.932805 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-455631ff-252b-4994-b105-d817dddfd7a4 tempest-VolumesListAdminTestJSON-1202706947 None] Created volume successfully. May 14 21:43:46.354543 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-00fbb041-169a-4b1a-8e72-b6c40bdb2ee1 req-7bf59908-177e-4076-8db5-b6c7d655083e tempest-VolumesBackupsTest-1778047115 None] Terminate volume connection completed successfully. May 14 21:43:46.354892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-00fbb041-169a-4b1a-8e72-b6c40bdb2ee1 req-7bf59908-177e-4076-8db5-b6c7d655083e tempest-VolumesBackupsTest-1778047115 None] Deleting attachment 34687cf4-b7bc-4316-ace6-e66ff7214212. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:43:47.739947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:43:47.763281 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Flow 'volume_create_manager' (5ab86edc-9e55-4e8b-8cd9-d3f222ae9c3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:47.771453 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3eec97e9-85d7-4383-9388-6b912c4f70b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:47.834044 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3eec97e9-85d7-4383-9388-6b912c4f70b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:43:47Z,deleted=False,deleted_at=None,display_description=None,display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=30b06ef6-71d4-46b9-9638-2660f73a5320,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e35ecebc92d248d684e05ad6ea30c8da',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:43:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:43:48Z,use_quota=True,user_id='35585104b2d04fdf9d447e2f34360122',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:47.842817 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7f3ba4fa-db0d-44ed-9a60-e633e59a4174) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:47.849000 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7f3ba4fa-db0d-44ed-9a60-e633e59a4174) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:47.855779 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (00f26b1c-cc2d-4e81-9896-bb0f48a17b2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:47.860045 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (00f26b1c-cc2d-4e81-9896-bb0f48a17b2c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '30b06ef6-71d4-46b9-9638-2660f73a5320', 'volume_name': 'volume-30b06ef6-71d4-46b9-9638-2660f73a5320', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:47.865624 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (21fb35ba-1e0d-4d9f-a563-ddb21b0b2e73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:47.872345 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (21fb35ba-1e0d-4d9f-a563-ddb21b0b2e73) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:47.880386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (16d9775c-7c5e-466d-83ff-2affc0ae52b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:47.882113 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Volume 30b06ef6-71d4-46b9-9638-2660f73a5320: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-30b06ef6-71d4-46b9-9638-2660f73a5320', 'volume_size': 1} May 14 21:43:47.882605 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] creating volume 'volume-30b06ef6-71d4-46b9-9638-2660f73a5320' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:43:47.883332 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:48.339039 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-42852489-8765-4ab4-bd84-813c67b69a1c tempest-GroupSnapshotsTest-385817527 None] group_snapshot 92f2c388-9126-4e29-8f4b-cc9c3066532e: created successfully May 14 21:43:48.418774 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:43:48.419164 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:43:48.424137 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (16d9775c-7c5e-466d-83ff-2affc0ae52b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-30b06ef6-71d4-46b9-9638-2660f73a5320', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:48.427658 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a8a8dae1-c941-44de-80de-c669f560f6c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:43:48.516802 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Volume volume-30b06ef6-71d4-46b9-9638-2660f73a5320 (30b06ef6-71d4-46b9-9638-2660f73a5320): created successfully May 14 21:43:48.521078 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a8a8dae1-c941-44de-80de-c669f560f6c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:43:48.528639 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Flow 'volume_create_manager' (5ab86edc-9e55-4e8b-8cd9-d3f222ae9c3f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:43:48.611636 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9b80406d-41be-46d2-b989-b5551c360436 tempest-VolumesListAdminTestJSON-534269106 None] Created volume successfully. May 14 21:43:49.064297 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] group_snapshot 92f2c388-9126-4e29-8f4b-cc9c3066532e: deleting May 14 21:43:49.082348 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] group_snapshot 92f2c388-9126-4e29-8f4b-cc9c3066532e: deleting {{(pid=123870) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4253}} May 14 21:43:49.098277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:49.826244 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] Lock "b'cinder-30b06ef6-71d4-46b9-9638-2660f73a5320-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:49.937638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:50.245853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:50.246531 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] Volume volume-30b06ef6-71d4-46b9-9638-2660f73a5320 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:50.434903 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] deleting rbd volume volume-30b06ef6-71d4-46b9-9638-2660f73a5320 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:50.448382 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] Lock "b'cinder-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:50.526736 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:50.680126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:50.680544 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] Volume volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:50.862894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] deleting rbd volume volume-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:50.982913 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-f2219050-2175-4ef5-b7e6-5497a584ba80-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:51.063769 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:53.462399 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] Volume volume-f2219050-2175-4ef5-b7e6-5497a584ba80 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:53.961228 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] deleting rbd volume volume-f2219050-2175-4ef5-b7e6-5497a584ba80 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:54.730539 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:56.977433 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] Created reservations ['c17ef989-ca04-4b6e-851e-f9e9cafaf429', '60d54e6d-09ab-42e1-8d8d-a367fb72dd17', '55c423a3-741a-495e-8239-3e97648d34cd', 'd14d5e60-dec6-4c58-ae98-ef499cb9dd62'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:56.999734 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] Created reservations ['445d967a-ad2e-4dcb-94e1-8ccaacea9c71', '23d6f128-25b0-441a-9654-7b952132421e', 'aa74b0a3-68dc-4946-96b3-8fd0aea8aca6', '73fe3394-4e72-4f12-9847-2fdba0f1770a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:57.191441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:57.213236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:57.287809 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:57.295271 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] Deleted volume successfully. May 14 21:43:57.392716 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-31ef0e85-f948-44c6-8d7e-00a293dec5d5 tempest-VolumesListAdminTestJSON-534269106 None] Lock "b'cinder-30b06ef6-71d4-46b9-9638-2660f73a5320-delete_volume'" released by "delete_volume" :: held 7.573s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:57.418206 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:57.549272 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:57.571859 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:43:57.576134 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] Deleted volume successfully. May 14 21:43:57.683993 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d3cb391f-1b36-4a75-93a1-b9da18680e0c tempest-VolumesListAdminTestJSON-1202706947 None] Lock "b'cinder-c43175d0-d11d-4cf2-8b0f-8e9a6a8ab4ed-delete_volume'" released by "delete_volume" :: held 7.236s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:43:57.723436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] Lock "b'cinder-b00305bf-f72d-407d-8936-6f45ba59ef2c-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:43:57.847388 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:43:58.280202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:43:58.280883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] Volume volume-b00305bf-f72d-407d-8936-6f45ba59ef2c is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:43:58.514880 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] deleting rbd volume volume-b00305bf-f72d-407d-8936-6f45ba59ef2c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:43:58.763409 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1510642463 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1510642463, it is now deprecated. Please use the default quota class for default quota. May 14 21:43:58.764300 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1510642463 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1510642463, it is now deprecated. Please use the default quota class for default quota. May 14 21:43:58.852051 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] Created reservations ['8385d402-0130-44c8-8ee2-f639b6db929f', '7f206cee-dd2f-4db4-a6e0-58f0f646d5d9', '0ccae4c0-9eec-4e38-8761-4d4de0b52040', 'b7292d75-a2b2-4bab-9623-153f4db681f4'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:59.149379 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] Created reservations ['2ef8379f-217e-4ba4-a158-82c947a6df03', 'be3c1f18-54a8-44c3-a7af-0d3d9c0e545d', 'c528e27d-60d3-4e75-b3bc-f29cee9fe037', '12bc0436-0322-48b4-b075-83d489d5348a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:43:59.417181 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0aeb5c17-248c-4b3a-985b-15176b184f4e tempest-GroupSnapshotsTest-385817527 None] group_snapshot 92f2c388-9126-4e29-8f4b-cc9c3066532e: deleted successfully May 14 21:44:00.006532 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:00.234133 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:44:00.234491 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Volume volume-17b5e099-759d-4436-a1c5-fea8e6ad081d is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:00.411987 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] deleting rbd volume volume-17b5e099-759d-4436-a1c5-fea8e6ad081d {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:44:00.588872 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] Created reservations ['fb9f1013-f96c-48b2-a55d-5f4286d2fee2', '13fd9b2f-51b5-4c16-a94c-0499c6d05707', 'ed16ed57-61f7-4a5a-9ff3-7fb50cc3becc', '4235a1bd-578f-4609-9364-08551ddb132f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:00.755273 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:00.776943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:00.800051 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:00.803992 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] Deleted volume successfully. May 14 21:44:00.862638 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-bd79e64b-820c-4390-858c-821dc7a86dfa tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-f2219050-2175-4ef5-b7e6-5497a584ba80-delete_volume'" released by "delete_volume" :: held 9.880s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:44:02.464558 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:44:02.485160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Flow 'volume_create_manager' (55099ca8-c18e-467c-9725-d3702af49d96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:02.491568 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ad77921d-2143-49df-97aa-90bc240490ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:02.506386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ad77921d-2143-49df-97aa-90bc240490ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:44:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1538230955',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=af532b40-a505-4682-982e-e62144e2824c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8526cad5fef54772a988df44138c90fe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:44:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:44:02Z,use_quota=True,user_id='80f409708620474aa128bf8412fa54be',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:02.511724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51921c45-621c-41b1-a983-12c75bbb3002) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:02.516902 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51921c45-621c-41b1-a983-12c75bbb3002) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:02.521162 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e2b6dac-56c4-42a2-a5fb-3809f9b5e62f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:02.600005 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e2b6dac-56c4-42a2-a5fb-3809f9b5e62f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'af532b40-a505-4682-982e-e62144e2824c', 'volume_name': 'volume-af532b40-a505-4682-982e-e62144e2824c', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:02.605178 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ee9035f-915f-449f-9903-f61d90ce0d50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:02.610269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ee9035f-915f-449f-9903-f61d90ce0d50) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:02.615008 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (436197c5-1db7-4255-b807-513f4cd72bed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:02.616338 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Volume af532b40-a505-4682-982e-e62144e2824c: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-af532b40-a505-4682-982e-e62144e2824c', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:44:02.616912 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Cloning af532b40-a505-4682-982e-e62144e2824c from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:44:02.627835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:44:02.691311 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 44, 2, 629148)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:44:02.691782 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:44:02.692180 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:44:02.696542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.071s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:44:02.699910 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:44:02.704484 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 44, 2, 702263)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:44:02.704896 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:44:02.705281 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:44:02.719916 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:02.994608 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] creating snapshot='volume-af532b40-a505-4682-982e-e62144e2824c.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:44:04.880857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-af532b40-a505-4682-982e-e62144e2824c.clone_snap' to 'volume-af532b40-a505-4682-982e-e62144e2824c' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:44:04.882695 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:44:04.883341 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:04.976181 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:04.997791 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:05.126392 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] Created reservations ['e3eb4ebe-8664-4935-808d-cf24faef4955', '0f25a331-560a-4196-b0d0-0baf247d978a', '9a87b447-405f-4058-b38c-9bef6666f370', '9f4cfb09-3064-4eeb-b345-b376c888a829'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:05.315330 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:05.342165 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:05.371260 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:05.377145 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] Deleted volume successfully. May 14 21:44:05.456439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-556a20c4-b03d-4536-8dfa-fe30a6f6847c tempest-VolumesListAdminTestJSON-1202706947 None] Lock "b'cinder-b00305bf-f72d-407d-8936-6f45ba59ef2c-delete_volume'" released by "delete_volume" :: held 7.733s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:44:06.342470 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] Lock "b'cinder-f87d52f8-7a10-48c9-a4e4-8edd88e288be-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:44:06.462457 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:06.714423 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:44:06.714845 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] Volume volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:06.949036 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] deleting rbd volume volume-f87d52f8-7a10-48c9-a4e4-8edd88e288be {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:44:06.956177 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:07.240002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:44:07.240352 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Volume volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:07.451449 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] deleting rbd volume volume-3b4fd2c5-6bd3-4f97-94d3-7ce1f7806f9e {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:44:07.497683 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:07.500109 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:44:07.500961 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:44:08.169918 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:44:08.170541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Marking volume af532b40-a505-4682-982e-e62144e2824c as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:44:08.257177 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to af532b40-a505-4682-982e-e62144e2824c. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:44:08.257531 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Creating volume glance metadata for volume af532b40-a505-4682-982e-e62144e2824c backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:44:08.432431 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (436197c5-1db7-4255-b807-513f4cd72bed) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-af532b40-a505-4682-982e-e62144e2824c', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:08.440799 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (51161e4e-3c32-462b-83dc-c622abc7dafd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:08.531213 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Volume volume-af532b40-a505-4682-982e-e62144e2824c (af532b40-a505-4682-982e-e62144e2824c): created successfully May 14 21:44:08.534903 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (51161e4e-3c32-462b-83dc-c622abc7dafd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:08.540820 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Flow 'volume_create_manager' (55099ca8-c18e-467c-9725-d3702af49d96) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:08.617193 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-6bf6cdb9-f0f4-4e9c-8db3-c0a84753523e tempest-VolumesBackupsTest-1778047115 None] Created volume successfully. May 14 21:44:10.327520 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:44:09Z,data_timestamp=2022-05-14T21:44:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-763507483',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=da61f1fe-5ccb-4511-9c48-da6e422121bd,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='8526cad5fef54772a988df44138c90fe',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='80f409708620474aa128bf8412fa54be',volume_id=af532b40-a505-4682-982e-e62144e2824c) May 14 21:44:10.328328 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] continue_backup in rpcapi backup_id da61f1fe-5ccb-4511-9c48-da6e422121bd {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:44:10.443571 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:44:11.091711 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.654s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:44:11.093370 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-af532b40-a505-4682-982e-e62144e2824c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'af532b40-a505-4682-982e-e62144e2824c', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:44:11.108743 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] Initialize volume connection completed successfully. May 14 21:44:13.320152 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] Created reservations ['f875dd8c-a20b-44b6-a220-fd95211c37ed', '09d3497d-b773-4756-98a8-57f01b55d6c2', '21da4f5c-6d36-47cb-b615-1f91d682042f', 'ffcc8e6f-a5d8-4f95-9584-7046a5867455'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:13.408224 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['fa97810e-0cfd-4214-bf39-600009011db1'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:13.432888 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1510642463 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1510642463, it is now deprecated. Please use the default quota class for default quota. May 14 21:44:13.487820 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:13.511506 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:13.534338 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:13.540898 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] Deleted volume successfully. May 14 21:44:13.633575 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['e23a02df-fabc-4dd9-ae92-7d5ec7bd2058', 'b5848757-8f63-456b-8f62-1837bea7b90e', '8bd05205-d72d-4785-819c-de344bd97ca4', 'a42f5696-e0ec-4fc2-b627-e7876bf6ae90'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:13.654012 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-7149e88a-a264-47d2-adf2-dc0311f9de6e tempest-VolumesListAdminTestJSON-1202706947 None] Lock "b'cinder-f87d52f8-7a10-48c9-a4e4-8edd88e288be-delete_volume'" released by "delete_volume" :: held 7.312s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:44:13.956063 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['25769319-32b7-434e-8781-b5fc7acd7554', '34236c72-a9d3-4b2d-b988-bcda3b1ce54c', 'd6da20fe-e775-453e-8534-d6f17d824576', '3560e9cd-0c20-43f5-8b8b-97fb4d124d2f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:14.351862 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:14.375568 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:14.400595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:14.404360 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3f33d09b-2449-46df-a250-48e86059b564 tempest-GroupSnapshotsTest-385817527 None] Delete group completed successfully. May 14 21:44:15.800429 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ef4dba8-324a-4ad0-b758-459a00040ca1 tempest-GroupSnapshotsTest-385817527 None] Group tempest-GroupSnapshotsTest-Group-1215594648: creating May 14 21:44:15.897901 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ef4dba8-324a-4ad0-b758-459a00040ca1 tempest-GroupSnapshotsTest-385817527 None] Group tempest-GroupSnapshotsTest-Group-1215594648: created successfully May 14 21:44:15.909649 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ef4dba8-324a-4ad0-b758-459a00040ca1 tempest-GroupSnapshotsTest-385817527 None] Create group completed successfully. May 14 21:44:17.790400 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:44:17.812720 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (7ba397a7-3643-4979-b978-6c514993b3b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:17.819909 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1edee5f0-60bb-41d9-b8f5-c85fe71e1106) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:17.836780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1edee5f0-60bb-41d9-b8f5-c85fe71e1106) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:44:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-5570607',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=52140ebe-e428-48bb-874b-3ab63a366e6d,host='ncdv-master-1@ceph#ceph',id=c313bf18-f66a-4a9a-bf41-ec9fcf212aa2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8a79a55f6b2f46ae89036ef9bb98639c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:44:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:44:18Z,use_quota=True,user_id='16ffd497e45e4683971ebee0eaebb975',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(a84f6c7c-2cee-4354-bbd0-735d5eb78e48),volume_type_id=a84f6c7c-2cee-4354-bbd0-735d5eb78e48)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:17.842632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bd74a0bc-5384-4348-a1d9-e6e6a26c9dd8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:17.847558 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bd74a0bc-5384-4348-a1d9-e6e6a26c9dd8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:17.853400 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e6a6ea28-21cf-4e93-985d-3196e7eb46a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:17.857071 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e6a6ea28-21cf-4e93-985d-3196e7eb46a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c313bf18-f66a-4a9a-bf41-ec9fcf212aa2', 'volume_name': 'volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:17.863042 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9b4b7071-2509-4905-9f99-890b154f8b40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:17.867388 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9b4b7071-2509-4905-9f99-890b154f8b40) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:17.873164 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (123fc6cc-f8b0-403b-87f2-9205dac522f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:17.878572 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Volume c313bf18-f66a-4a9a-bf41-ec9fcf212aa2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2', 'volume_size': 1} May 14 21:44:17.879241 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] creating volume 'volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:44:17.880149 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:19.647561 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:44:19.667170 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Flow 'volume_create_manager' (43e2d0b0-f54d-43b4-9958-9b4773dbb39c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:19.674437 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (95c9e28b-1548-4583-bc06-8e78df07dc3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:19.691134 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (95c9e28b-1548-4583-bc06-8e78df07dc3f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:44:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-615543322',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=31253405-2458-41a9-977a-91f08eaa7afb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d625ce83e9114cfaae698d73b917709d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:44:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:44:19Z,use_quota=True,user_id='0860ddc4bdec438aba103113979deb07',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:19.696673 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (15fc248e-de9b-4ad7-9c24-b97d34211dec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:19.700796 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (15fc248e-de9b-4ad7-9c24-b97d34211dec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:19.707253 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b68039a6-1270-4b5f-a2e8-c718ecebea27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:19.711241 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b68039a6-1270-4b5f-a2e8-c718ecebea27) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '31253405-2458-41a9-977a-91f08eaa7afb', 'volume_name': 'volume-31253405-2458-41a9-977a-91f08eaa7afb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:19.716228 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (40267388-690e-4999-8498-36648b04776d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:19.720983 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (40267388-690e-4999-8498-36648b04776d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:19.725607 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0a1bf5c-b489-4993-aa65-33373c641a9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:19.726667 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Volume 31253405-2458-41a9-977a-91f08eaa7afb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-31253405-2458-41a9-977a-91f08eaa7afb', 'volume_size': 1} May 14 21:44:19.727053 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] creating volume 'volume-31253405-2458-41a9-977a-91f08eaa7afb' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:44:19.727728 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:20.364997 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:44:20.365486 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:44:20.370235 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (123fc6cc-f8b0-403b-87f2-9205dac522f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:20.375740 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (569e16f4-87c7-4628-a89f-68622c8e3ccc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:20.446102 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Volume volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2 (c313bf18-f66a-4a9a-bf41-ec9fcf212aa2): created successfully May 14 21:44:20.449763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (569e16f4-87c7-4628-a89f-68622c8e3ccc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:20.456142 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Flow 'volume_create_manager' (7ba397a7-3643-4979-b978-6c514993b3b9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:20.537518 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-74da7f68-e732-426c-8d15-76c70240c00f tempest-GroupSnapshotsTest-385817527 None] Created volume successfully. May 14 21:44:21.171540 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:44:21.172094 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:44:21.177109 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0a1bf5c-b489-4993-aa65-33373c641a9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-31253405-2458-41a9-977a-91f08eaa7afb', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:21.182246 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a36c8475-ca03-45ea-8a58-d1b496dff1b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:21.257233 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Volume volume-31253405-2458-41a9-977a-91f08eaa7afb (31253405-2458-41a9-977a-91f08eaa7afb): created successfully May 14 21:44:21.260927 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a36c8475-ca03-45ea-8a58-d1b496dff1b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:21.266509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Flow 'volume_create_manager' (43e2d0b0-f54d-43b4-9958-9b4773dbb39c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:21.341911 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4642001f-0760-4ba6-88cf-51cdf6333071 tempest-VolumesTransfersTest-234608367 None] Created volume successfully. May 14 21:44:21.384110 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7faec2b8-0ef6-4e26-9be2-fcbcf45fa576 tempest-GroupSnapshotsTest-385817527 None] GroupSnapshot 15317636-9702-4190-b922-46fa07b13023: creating. May 14 21:44:21.407920 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7faec2b8-0ef6-4e26-9be2-fcbcf45fa576 tempest-GroupSnapshotsTest-385817527 None] Group snapshot 15317636-9702-4190-b922-46fa07b13023: creating. {{(pid=123870) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4078}} May 14 21:44:21.426914 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7faec2b8-0ef6-4e26-9be2-fcbcf45fa576 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:22.757596 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8c02cac4-a1e9-4f40-8db2-c2a27f4b7ec7 tempest-VolumesTransfersTest-1628107971 None] Transfer volume completed successfully. May 14 21:44:23.801293 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] Lock "b'cinder-31253405-2458-41a9-977a-91f08eaa7afb-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:44:23.892863 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:24.531357 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:44:24.531889 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] Volume volume-31253405-2458-41a9-977a-91f08eaa7afb is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:24.755510 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] deleting rbd volume volume-31253405-2458-41a9-977a-91f08eaa7afb {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:44:27.390151 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7faec2b8-0ef6-4e26-9be2-fcbcf45fa576 tempest-GroupSnapshotsTest-385817527 None] group_snapshot 15317636-9702-4190-b922-46fa07b13023: created successfully May 14 21:44:27.973661 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] group_snapshot 15317636-9702-4190-b922-46fa07b13023: deleting May 14 21:44:28.002172 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] group_snapshot 15317636-9702-4190-b922-46fa07b13023: deleting {{(pid=123870) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4253}} May 14 21:44:28.025252 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:34.291344 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-412438951 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-412438951, it is now deprecated. Please use the default quota class for default quota. May 14 21:44:34.292174 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-412438951 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-412438951, it is now deprecated. Please use the default quota class for default quota. May 14 21:44:34.374337 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] Created reservations ['7dc524b6-c199-49d7-8e6e-8c4155a8a88e', 'f0f87838-23ec-466b-b8ac-e283bb701757', '17718c55-d520-4e82-8b8b-453590b5a87f', '02ec2763-9bcc-4700-a30b-df6b1e82d945'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:34.605337 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7ca6204d-f702-42d5-9e9a-0cf663e9d4ad tempest-GroupSnapshotsTest-385817527 None] group_snapshot 15317636-9702-4190-b922-46fa07b13023: deleted successfully May 14 21:44:35.775075 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:36.013187 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:44:36.013533 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] Volume volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:36.361353 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] deleting rbd volume volume-c313bf18-f66a-4a9a-bf41-ec9fcf212aa2 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:44:36.518125 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] Created reservations ['5af4b2ae-4fa1-4447-a875-9363853bc8bf', '82eeff77-63ca-402d-b7d9-0b52a5cd32da', '38166dea-240b-4312-987f-4e9cd70747c2', '18eec134-b0ab-4a82-9b6b-7bc888d11115'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:36.714439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:36.738852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:36.762692 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:36.768060 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] Deleted volume successfully. May 14 21:44:36.848514 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4f62227d-9b16-4c6c-a5d9-434dbdda9c87 tempest-VolumesTransfersTest-61980726 None] Lock "b'cinder-31253405-2458-41a9-977a-91f08eaa7afb-delete_volume'" released by "delete_volume" :: held 13.039s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:44:38.493004 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:44:38.512739 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Flow 'volume_create_manager' (379f9e74-f4c4-4ffa-af18-f971d4dc4838) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:38.519370 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (eaae846c-9b8e-4512-84e3-c4586fd499d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:38.539904 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (eaae846c-9b8e-4512-84e3-c4586fd499d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:44:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-1571044843',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=35a47d2e-8654-4981-958e-fc0c2e0db45a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d625ce83e9114cfaae698d73b917709d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:44:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:44:38Z,use_quota=True,user_id='0860ddc4bdec438aba103113979deb07',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:38.545472 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2051a139-ce69-40ed-a108-fbdc344973f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:38.550263 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2051a139-ce69-40ed-a108-fbdc344973f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:38.555411 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2475477c-0718-443d-87ba-56b0bacbab78) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:38.559517 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2475477c-0718-443d-87ba-56b0bacbab78) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '35a47d2e-8654-4981-958e-fc0c2e0db45a', 'volume_name': 'volume-35a47d2e-8654-4981-958e-fc0c2e0db45a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:38.564940 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e94d04d8-18b4-4d9b-a3f2-dc5d5ba7ac1c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:38.568952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e94d04d8-18b4-4d9b-a3f2-dc5d5ba7ac1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:38.574814 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ec55de2a-d224-4da1-a523-ac3bd4268253) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:38.575995 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Volume 35a47d2e-8654-4981-958e-fc0c2e0db45a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-35a47d2e-8654-4981-958e-fc0c2e0db45a', 'volume_size': 1} May 14 21:44:38.576407 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] creating volume 'volume-35a47d2e-8654-4981-958e-fc0c2e0db45a' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:44:38.577217 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:40.216279 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:44:40.216934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:44:40.220977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ec55de2a-d224-4da1-a523-ac3bd4268253) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-35a47d2e-8654-4981-958e-fc0c2e0db45a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:40.225787 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fec27036-300c-41a0-96a5-d4d22068384e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:44:40.309563 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Volume volume-35a47d2e-8654-4981-958e-fc0c2e0db45a (35a47d2e-8654-4981-958e-fc0c2e0db45a): created successfully May 14 21:44:40.313316 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fec27036-300c-41a0-96a5-d4d22068384e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:44:40.318952 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Flow 'volume_create_manager' (379f9e74-f4c4-4ffa-af18-f971d4dc4838) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:44:40.416898 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9baaa1b5-65bb-4e9b-8f12-87cb789f1d98 tempest-VolumesTransfersTest-234608367 None] Created volume successfully. May 14 21:44:42.141118 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] Lock "b'cinder-35a47d2e-8654-4981-958e-fc0c2e0db45a-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:44:42.246264 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:42.450915 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:44:42.451959 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] Volume volume-35a47d2e-8654-4981-958e-fc0c2e0db45a is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:44:42.772002 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] deleting rbd volume volume-35a47d2e-8654-4981-958e-fc0c2e0db45a {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:44:47.833974 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['407230a1-ac44-41b8-a913-91073c69e889'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:47.858890 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-412438951 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-412438951, it is now deprecated. Please use the default quota class for default quota. May 14 21:44:47.948639 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] Created reservations ['ae252844-f85c-470b-aca5-4978029324ce', 'adfc6b63-cd9b-4e8b-9cd8-3a435d2db91a', '0a0a4288-d5e0-40ff-84f8-91a030010ebc', '11508779-99d2-4c50-87ae-9cf1b4db2f8c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:48.313137 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:48.358507 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:48.466022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:48.470857 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-590ff0ca-64bd-4c93-8524-a00e53a11c02 tempest-GroupSnapshotsTest-385817527 None] Delete group completed successfully. May 14 21:44:53.685204 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] Created reservations ['eac2d5ba-5035-4b23-aeee-5dcd48e70172', '50769692-2507-4db2-9e01-84fe265d9e06', 'b15d8e57-1e57-4d0c-9603-b387a4b791b1', 'eb07be20-3320-4cdc-9315-282321b83cfd'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:44:53.885626 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:53.912213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:44:53.952099 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:44:53.956947 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] Deleted volume successfully. May 14 21:44:54.159300 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-017a6a1b-ab0a-4f85-8e3d-b02a42ff7ed9 tempest-VolumesTransfersTest-61980726 None] Lock "b'cinder-35a47d2e-8654-4981-958e-fc0c2e0db45a-delete_volume'" released by "delete_volume" :: held 12.018s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:45:00.658416 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:45:00.677934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Flow 'volume_create_manager' (b4b77891-da68-4d7e-a80c-b91f44594b8f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:00.686693 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (995a9955-b646-4e88-8e7a-a9b97613ec63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:00.700344 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (995a9955-b646-4e88-8e7a-a9b97613ec63) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:45:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-121826687',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=22b0a87f-6086-493e-adf8-5e0b48222bf7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5ea54abe27dd44beb76e2876a4cb53fd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:45:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:45:00Z,use_quota=True,user_id='8c023bd546014ceda5e43ba1c93ca7b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:00.705908 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (313b9b75-be8a-4c68-80c3-43099f65dc15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:00.710230 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (313b9b75-be8a-4c68-80c3-43099f65dc15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:00.715516 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ca133c9-84b4-4f7a-a4ce-2a712a82ed1e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:00.813821 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ca133c9-84b4-4f7a-a4ce-2a712a82ed1e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '22b0a87f-6086-493e-adf8-5e0b48222bf7', 'volume_name': 'volume-22b0a87f-6086-493e-adf8-5e0b48222bf7', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:00.819698 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fd150dcd-017e-432c-a768-14bb87d625f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:00.824845 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fd150dcd-017e-432c-a768-14bb87d625f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:00.835932 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c24caf70-64ab-4122-886d-b3f33fc5583a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:00.835932 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Volume 22b0a87f-6086-493e-adf8-5e0b48222bf7: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-22b0a87f-6086-493e-adf8-5e0b48222bf7', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:45:00.836708 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Cloning 22b0a87f-6086-493e-adf8-5e0b48222bf7 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:45:00.840486 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.007s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:45:00.919018 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 45, 0, 844391)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:45:00.919462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:45:00.919917 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:45:00.925652 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.085s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:45:00.929822 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:45:00.933275 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 45, 0, 930678)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:45:00.933684 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:45:00.934416 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:45:00.945722 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:01.181991 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] creating snapshot='volume-22b0a87f-6086-493e-adf8-5e0b48222bf7.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:45:03.428089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-22b0a87f-6086-493e-adf8-5e0b48222bf7.clone_snap' to 'volume-22b0a87f-6086-493e-adf8-5e0b48222bf7' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:45:03.445006 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7333af3b-d8ca-4e45-bfdf-bb736d793d1c tempest-GroupsTest-1793587503 None] Group tempest-Group1-1860152419: creating May 14 21:45:03.453192 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] Terminate volume connection completed successfully. May 14 21:45:03.474339 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c67c3e81-7926-4cd4-a64c-737fb132440d tempest-VolumesBackupsTest-1778047115 None] Remove volume export completed successfully. May 14 21:45:03.619261 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7333af3b-d8ca-4e45-bfdf-bb736d793d1c tempest-GroupsTest-1793587503 None] Group tempest-Group1-1860152419: created successfully May 14 21:45:03.628243 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7333af3b-d8ca-4e45-bfdf-bb736d793d1c tempest-GroupsTest-1793587503 None] Create group completed successfully. May 14 21:45:04.576866 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:45:04.578052 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:45:04.578612 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:45:04.889368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:45:04.890109 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Marking volume 22b0a87f-6086-493e-adf8-5e0b48222bf7 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:45:04.898787 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:45:04.900034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:04.934794 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:04.962330 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:45:04.990785 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fdeececb-bd80-44a4-a9f0-7450ebb9da3c tempest-GroupsTest-1793587503 None] Group tempest-Group2-1835439619: creating May 14 21:45:05.107940 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 22b0a87f-6086-493e-adf8-5e0b48222bf7. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:45:05.108297 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Creating volume glance metadata for volume 22b0a87f-6086-493e-adf8-5e0b48222bf7 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:45:05.137106 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:45:05.158635 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Flow 'volume_create_manager' (dce3dc26-ae97-4fd5-a7ca-3b51de8521e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:05.165403 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9b55c0ab-5b31-40d7-8bba-7285a3842a5d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.183677 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9b55c0ab-5b31-40d7-8bba-7285a3842a5d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:45:04Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_da61f1fe-5ccb-4511-9c48-da6e422121bd',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fe2b0773-b648-40ed-8148-08fa2937605a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8526cad5fef54772a988df44138c90fe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:45:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:45:05Z,use_quota=True,user_id='80f409708620474aa128bf8412fa54be',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.190243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1d4ef4ba-d21a-416a-83e4-cb07b07c0233) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.194697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1d4ef4ba-d21a-416a-83e4-cb07b07c0233) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.199632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f715ccfd-ad4a-4a4d-a666-6347a83fcc57) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.203690 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f715ccfd-ad4a-4a4d-a666-6347a83fcc57) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fe2b0773-b648-40ed-8148-08fa2937605a', 'volume_name': 'volume-fe2b0773-b648-40ed-8148-08fa2937605a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.209879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (087bc5fe-7792-476a-87fa-defcd82a0166) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.213781 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (087bc5fe-7792-476a-87fa-defcd82a0166) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.218501 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fd455029-387f-4b75-a84a-a992747e506b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.219597 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Volume fe2b0773-b648-40ed-8148-08fa2937605a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fe2b0773-b648-40ed-8148-08fa2937605a', 'volume_size': 1} May 14 21:45:05.220555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] creating volume 'volume-fe2b0773-b648-40ed-8148-08fa2937605a' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:45:05.221365 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:05.245007 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fdeececb-bd80-44a4-a9f0-7450ebb9da3c tempest-GroupsTest-1793587503 None] Group tempest-Group2-1835439619: created successfully May 14 21:45:05.256264 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-fdeececb-bd80-44a4-a9f0-7450ebb9da3c tempest-GroupsTest-1793587503 None] Create group completed successfully. May 14 21:45:05.385634 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c24caf70-64ab-4122-886d-b3f33fc5583a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-22b0a87f-6086-493e-adf8-5e0b48222bf7', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.390653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c55665f-a814-40d5-8def-665fe67ee2fd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.519294 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Volume volume-22b0a87f-6086-493e-adf8-5e0b48222bf7 (22b0a87f-6086-493e-adf8-5e0b48222bf7): created successfully May 14 21:45:05.524120 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c55665f-a814-40d5-8def-665fe67ee2fd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.529000 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Flow 'volume_create_manager' (b4b77891-da68-4d7e-a80c-b91f44594b8f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:05.627979 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3c509441-6376-409a-951c-0c786c6be104 tempest-VolumesExtendTest-1486560883 None] Created volume successfully. May 14 21:45:05.813817 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:45:05.814594 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:45:05.818422 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fd455029-387f-4b75-a84a-a992747e506b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fe2b0773-b648-40ed-8148-08fa2937605a', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.823636 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2dcd80b-bc0b-48dd-bd78-693274768b98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:05.896616 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Volume volume-fe2b0773-b648-40ed-8148-08fa2937605a (fe2b0773-b648-40ed-8148-08fa2937605a): created successfully May 14 21:45:05.900437 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2dcd80b-bc0b-48dd-bd78-693274768b98) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:05.905951 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Flow 'volume_create_manager' (dce3dc26-ae97-4fd5-a7ca-3b51de8521e5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:06.011637 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Created volume successfully. May 14 21:45:06.030932 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7f3ecac8-bcd0-42dd-9fb6-5d690623f995 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:06.943040 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:45:07.072632 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7f3ecac8-bcd0-42dd-9fb6-5d690623f995 tempest-VolumesExtendTest-1486560883 None] Extend volume from 1 GB to 2 GB. {{(pid=123870) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1880}} May 14 21:45:07.096627 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:45:07.135021 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Flow 'volume_create_manager' (145e5900-5d10-4211-99a6-ae8b8bced843) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:07.152173 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7286061-f2cb-4009-9001-5195d3bcd4a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:07.192217 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7286061-f2cb-4009-9001-5195d3bcd4a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:45:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-volume-1793094014',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=9207aeb0-2f9c-41d0-a7b6-f60aafaabcc9,host='ncdv-master-1@ceph#ceph',id=69b72427-ff4f-40ee-873f-cc4f29b76cc1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='70d62c44e62f4f908188af21b5653cc7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:45:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:45:07Z,use_quota=True,user_id='c18d326a4dd84ac3aa1cabb4ea37a644',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(b38055ab-5ab0-47e6-8fc1-04d242dd5c0f),volume_type_id=b38055ab-5ab0-47e6-8fc1-04d242dd5c0f)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:07.199548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cc24ab25-552b-4f9a-be46-387d6778e766) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:07.203944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cc24ab25-552b-4f9a-be46-387d6778e766) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:07.209823 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb2dd4b2-686b-4f79-9b46-99b4a8ce33b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:07.214329 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb2dd4b2-686b-4f79-9b46-99b4a8ce33b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '69b72427-ff4f-40ee-873f-cc4f29b76cc1', 'volume_name': 'volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:07.225932 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d5280e02-f619-4b9e-b65f-0842778e9656) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:07.230800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d5280e02-f619-4b9e-b65f-0842778e9656) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:07.237885 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0cce1e17-4c46-4b07-9c3e-5ffbd96c42c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:07.241166 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Volume 69b72427-ff4f-40ee-873f-cc4f29b76cc1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1', 'volume_size': 1} May 14 21:45:07.241663 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] creating volume 'volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:45:07.242593 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:07.364983 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7f3ecac8-bcd0-42dd-9fb6-5d690623f995 tempest-VolumesExtendTest-1486560883 None] Extend volume completed successfully. May 14 21:45:07.708257 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.765s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:45:07.709130 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-fe2b0773-b648-40ed-8148-08fa2937605a', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'fe2b0773-b648-40ed-8148-08fa2937605a', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:45:07.729925 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Initialize volume connection completed successfully. May 14 21:45:08.008980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:45:08.010335 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:45:08.014550 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0cce1e17-4c46-4b07-9c3e-5ffbd96c42c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:08.021049 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3bd1b007-76e6-4c16-aab7-680890ddf999) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:08.136733 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Volume volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1 (69b72427-ff4f-40ee-873f-cc4f29b76cc1): created successfully May 14 21:45:08.140862 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3bd1b007-76e6-4c16-aab7-680890ddf999) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:08.147144 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Flow 'volume_create_manager' (145e5900-5d10-4211-99a6-ae8b8bced843) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:08.245726 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1be00f00-a66b-4baa-b9e6-ac66c5901633 tempest-GroupsTest-1793587503 None] Created volume successfully. May 14 21:45:08.902025 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:45:08.944236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Flow 'volume_create_manager' (70b69a03-80e0-49b6-ac76-3fe8aa8c4f26) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:08.951944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5043fe91-cd93-4a17-8e32-0ebef76eff44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:08.977154 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5043fe91-cd93-4a17-8e32-0ebef76eff44) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:45:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-1492383697',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=580b963c-7f54-4166-afb5-3e1d9e0b2c7e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5ea54abe27dd44beb76e2876a4cb53fd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:45:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:45:09Z,use_quota=True,user_id='8c023bd546014ceda5e43ba1c93ca7b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:08.984873 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (28ad4dee-a662-4aa2-9d31-0470cdf3a8eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:09.001853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (28ad4dee-a662-4aa2-9d31-0470cdf3a8eb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:09.008476 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3e352ad-4228-48dc-bd9a-751d26acfd3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:09.013564 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3e352ad-4228-48dc-bd9a-751d26acfd3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '580b963c-7f54-4166-afb5-3e1d9e0b2c7e', 'volume_name': 'volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:09.019440 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (30e51fdc-9bae-42a3-b74d-bf021d23b488) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:09.028894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (30e51fdc-9bae-42a3-b74d-bf021d23b488) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:09.041766 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (59ff184b-120d-4ee7-bab4-b63fb9ee6975) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:09.043014 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Volume 580b963c-7f54-4166-afb5-3e1d9e0b2c7e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e', 'volume_size': 1} May 14 21:45:09.043503 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] creating volume 'volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:45:09.045016 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:09.693390 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:10.143274 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:45:10.143274 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Volume volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:45:10.417737 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] deleting rbd volume volume-69b72427-ff4f-40ee-873f-cc4f29b76cc1 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:45:10.428850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:45:10.428850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:45:10.432776 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (59ff184b-120d-4ee7-bab4-b63fb9ee6975) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:10.440824 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (079b9173-1de6-4810-8401-9c3f06cc80d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:10.513324 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Volume volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e (580b963c-7f54-4166-afb5-3e1d9e0b2c7e): created successfully May 14 21:45:10.519272 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (079b9173-1de6-4810-8401-9c3f06cc80d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:10.525499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Flow 'volume_create_manager' (70b69a03-80e0-49b6-ac76-3fe8aa8c4f26) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:10.627875 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-88f2eda1-1454-45f6-9532-81e836738bcf tempest-VolumesExtendTest-1486560883 None] Created volume successfully. May 14 21:45:11.473980 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-212f1192-c59f-4869-ad9b-cd0c36ab496a tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:17.021617 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-212f1192-c59f-4869-ad9b-cd0c36ab496a tempest-VolumesExtendTest-1486560883 None] Create snapshot completed successfully May 14 21:45:17.770567 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1390a11c-05e0-4779-bd45-82d897d6a676 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:19.987833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Created reservations ['41bc0550-4d16-4fc6-ae23-2d43e519f6bb'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:20.006597 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-1069408755 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-1069408755, it is now deprecated. Please use the default quota class for default quota. May 14 21:45:20.007395 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-1069408755 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-1069408755, it is now deprecated. Please use the default quota class for default quota. May 14 21:45:20.067989 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Created reservations ['6ad07faf-b368-4d57-9f8a-faed8dcbaeb5', '1426a759-194a-4f5e-89ff-cfb86f32ede0', '1a48f831-db07-4ae2-ac2d-4323b0752e4b', 'a6a64469-f603-42c1-8c25-5191134a8038'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:20.172305 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-1390a11c-05e0-4779-bd45-82d897d6a676 tempest-VolumesExtendTest-1486560883 None] Extend volume from 1 GB to 2 GB. {{(pid=123870) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1880}} May 14 21:45:20.405052 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-1390a11c-05e0-4779-bd45-82d897d6a676 tempest-VolumesExtendTest-1486560883 None] Extend volume completed successfully. May 14 21:45:20.426015 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:20.451307 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:20.479268 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:45:20.483224 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-aa462040-bade-4871-9f23-efc5c4085d96 tempest-GroupsTest-1793587503 None] Delete group completed successfully. May 14 21:45:20.730409 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-05ad4989-5746-4f3b-ba40-7743764f523f tempest-GroupsTest-1793587503 None] Created reservations ['fb3d1962-2838-4623-9d57-9528fc85bdb6'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:20.910420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05ad4989-5746-4f3b-ba40-7743764f523f tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:20.943279 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05ad4989-5746-4f3b-ba40-7743764f523f tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:20.971462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-05ad4989-5746-4f3b-ba40-7743764f523f tempest-GroupsTest-1793587503 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:45:20.975416 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-05ad4989-5746-4f3b-ba40-7743764f523f tempest-GroupsTest-1793587503 None] Delete group completed successfully. May 14 21:45:21.143352 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4465c537-5808-496f-86cb-f39189ee3576 tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-52b04f18-5b42-412f-ab48-7aab3ef3caca-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:45:21.145137 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-4465c537-5808-496f-86cb-f39189ee3576 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:22.672541 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-292c989a-0726-4402-adef-eff6b771dc20 tempest-GroupsTest-1793587503 None] Group tempest-GroupsTest-Group-432861194: creating May 14 21:45:22.786126 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-292c989a-0726-4402-adef-eff6b771dc20 tempest-GroupsTest-1793587503 None] Group tempest-GroupsTest-Group-432861194: created successfully May 14 21:45:22.794737 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-292c989a-0726-4402-adef-eff6b771dc20 tempest-GroupsTest-1793587503 None] Create group completed successfully. May 14 21:45:24.538667 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:45:24.561933 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Flow 'volume_create_manager' (eb8bffba-35fe-4fec-8036-5d1e296dbbda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:24.569666 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a312bc5f-e1d2-4f4f-9300-9c1901fc2510) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:24.586942 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a312bc5f-e1d2-4f4f-9300-9c1901fc2510) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:45:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1297591270',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=44ccc4a4-bcc8-4f4a-8e90-7ea3e21d6916,host='ncdv-master-1@ceph#ceph',id=7af23272-7214-4f16-ad85-e41a69207fa5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='70d62c44e62f4f908188af21b5653cc7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:45:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:45:24Z,use_quota=True,user_id='c18d326a4dd84ac3aa1cabb4ea37a644',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(eaed0326-aa0e-439f-abd9-5a321eb5eec3),volume_type_id=eaed0326-aa0e-439f-abd9-5a321eb5eec3)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:24.592658 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (41b8d574-aba3-4ab7-a726-cd9ecf6b6a61) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:24.597539 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (41b8d574-aba3-4ab7-a726-cd9ecf6b6a61) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:24.602837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8131b2c8-964c-42a7-8e81-96a78b434f02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:24.607129 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8131b2c8-964c-42a7-8e81-96a78b434f02) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7af23272-7214-4f16-ad85-e41a69207fa5', 'volume_name': 'volume-7af23272-7214-4f16-ad85-e41a69207fa5', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:24.612857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82177f96-7b74-4703-b627-e159adc6d9d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:24.616878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82177f96-7b74-4703-b627-e159adc6d9d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:24.621653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c265d1a0-c860-4e20-bf10-4b45962bc593) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:24.622769 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Volume 7af23272-7214-4f16-ad85-e41a69207fa5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7af23272-7214-4f16-ad85-e41a69207fa5', 'volume_size': 1} May 14 21:45:24.623162 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] creating volume 'volume-7af23272-7214-4f16-ad85-e41a69207fa5' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:45:24.623976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:26.524294 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:45:26.524996 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:45:26.528588 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c265d1a0-c860-4e20-bf10-4b45962bc593) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7af23272-7214-4f16-ad85-e41a69207fa5', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:26.533229 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2684f92f-b1d2-494c-afa6-3ccb34df1e82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:26.618595 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Volume volume-7af23272-7214-4f16-ad85-e41a69207fa5 (7af23272-7214-4f16-ad85-e41a69207fa5): created successfully May 14 21:45:26.623606 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2684f92f-b1d2-494c-afa6-3ccb34df1e82) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:26.629199 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Flow 'volume_create_manager' (eb8bffba-35fe-4fec-8036-5d1e296dbbda) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:26.707322 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-149cf329-4b68-487d-9000-30e7e2b1f5fa tempest-GroupsTest-1793587503 None] Created volume successfully. May 14 21:45:26.731461 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-4465c537-5808-496f-86cb-f39189ee3576 tempest-VolumesExtendTest-1486560883 None] Created reservations ['5b30df14-9e96-4b55-b91f-dec1e4c17e47', 'c6ac8f70-aec5-48a3-a53f-6ada8e80b6a7', '60fcf184-2316-4034-a98a-5ecd2f081d68', '6189cc3f-b873-4455-bce3-a97bf0f1a19a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:26.884914 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-4465c537-5808-496f-86cb-f39189ee3576 tempest-VolumesExtendTest-1486560883 None] Delete snapshot completed successfully. May 14 21:45:26.889382 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-4465c537-5808-496f-86cb-f39189ee3576 tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-52b04f18-5b42-412f-ab48-7aab3ef3caca-delete_snapshot'" released by "delete_snapshot" :: held 5.746s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:45:27.578960 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-580b963c-7f54-4166-afb5-3e1d9e0b2c7e-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:45:27.685243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:28.027049 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:45:28.027640 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] Volume volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:45:28.387398 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] deleting rbd volume volume-580b963c-7f54-4166-afb5-3e1d9e0b2c7e {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:45:28.537116 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:45:28.556919 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Flow 'volume_create_manager' (41e2dfa3-88d7-4789-b33b-37580bbf9a87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:28.565264 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (40e3e2d8-a606-4cde-9e79-4916b78dae07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:28.583450 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (40e3e2d8-a606-4cde-9e79-4916b78dae07) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:45:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-125520996',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=44ccc4a4-bcc8-4f4a-8e90-7ea3e21d6916,host='ncdv-master-1@ceph#ceph',id=34499934-5593-48de-bbc7-87814ea38fed,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='70d62c44e62f4f908188af21b5653cc7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:45:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:45:28Z,use_quota=True,user_id='c18d326a4dd84ac3aa1cabb4ea37a644',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(eaed0326-aa0e-439f-abd9-5a321eb5eec3),volume_type_id=eaed0326-aa0e-439f-abd9-5a321eb5eec3)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:28.588909 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0020f8e4-0e65-43da-b8b3-303e1a52d9cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:28.593097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0020f8e4-0e65-43da-b8b3-303e1a52d9cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:28.598865 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7cfe7f41-c962-41f7-b0df-fec7fb65e967) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:28.603304 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7cfe7f41-c962-41f7-b0df-fec7fb65e967) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '34499934-5593-48de-bbc7-87814ea38fed', 'volume_name': 'volume-34499934-5593-48de-bbc7-87814ea38fed', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:28.608592 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c6aa5f00-58f7-4288-b28d-4e850a956ac0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:28.612662 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c6aa5f00-58f7-4288-b28d-4e850a956ac0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:28.617418 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7c7f8e75-ed71-416e-a7d0-f718c5618b00) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:28.618612 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Volume 34499934-5593-48de-bbc7-87814ea38fed: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-34499934-5593-48de-bbc7-87814ea38fed', 'volume_size': 1} May 14 21:45:28.619072 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] creating volume 'volume-34499934-5593-48de-bbc7-87814ea38fed' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:45:28.619841 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:29.635029 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:45:29.635751 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:45:29.639585 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7c7f8e75-ed71-416e-a7d0-f718c5618b00) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-34499934-5593-48de-bbc7-87814ea38fed', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:29.645160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (083c813d-24b2-41ca-a015-b3fffaa41651) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:45:29.735811 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Volume volume-34499934-5593-48de-bbc7-87814ea38fed (34499934-5593-48de-bbc7-87814ea38fed): created successfully May 14 21:45:29.740198 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (083c813d-24b2-41ca-a015-b3fffaa41651) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:45:29.745873 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Flow 'volume_create_manager' (41e2dfa3-88d7-4789-b33b-37580bbf9a87) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:45:29.821960 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3e34cdc7-5a88-4d88-a533-2bc5740bd0d6 tempest-GroupsTest-1793587503 None] Created volume successfully. May 14 21:45:30.654914 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-de12d5e7-1e3a-4635-8385-7851fd7a1b59 tempest-GroupsTest-1793587503 None] Update group completed successfully. May 14 21:45:32.003333 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0903a5c8-f5fc-4717-8373-32fe2f1b7509 tempest-GroupsTest-1793587503 None] Update group completed successfully. May 14 21:45:33.444089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:33.838759 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:45:33.839291 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Volume volume-34499934-5593-48de-bbc7-87814ea38fed is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:45:34.233406 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] deleting rbd volume volume-34499934-5593-48de-bbc7-87814ea38fed {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:45:39.661214 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] Created reservations ['db89a957-3ba8-4d56-9c6a-873a897e2997', 'cade1c93-9c86-4b61-923a-2d59d56f6ddc', '1d2a9dd5-c415-4fe8-833e-e40243ae40a5', 'a72530b9-45ad-48df-b5ef-612db21930f3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:39.805101 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:39.828308 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:39.855332 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:45:39.860804 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] Deleted volume successfully. May 14 21:45:39.973811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-ddf8eb72-250d-4d7e-8829-48588d760137 tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-580b963c-7f54-4166-afb5-3e1d9e0b2c7e-delete_volume'" released by "delete_volume" :: held 12.395s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:45:40.370256 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-22b0a87f-6086-493e-adf8-5e0b48222bf7-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:45:40.487401 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:40.900880 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:45:41.125574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] deleting rbd volume volume-22b0a87f-6086-493e-adf8-5e0b48222bf7 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:45:44.220378 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:44.489564 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:45:44.490668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Volume volume-7af23272-7214-4f16-ad85-e41a69207fa5 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:45:44.724681 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] deleting rbd volume volume-7af23272-7214-4f16-ad85-e41a69207fa5 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:45:49.257727 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:45:49.628004 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] deleting parent snapshot volume-22b0a87f-6086-493e-adf8-5e0b48222bf7.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:45:52.787090 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] Created reservations ['89b029d6-0832-484a-ae37-657bd4745957', '01b64b1a-4ce3-481a-b694-e1495fc1a690', 'b37f9756-35eb-45ef-9398-aeae778ba922', 'f028fe49-7a9d-4844-9dff-474d70e9226f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:52.908610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Created reservations ['b9c037f5-dfd1-41cd-a92a-7501f1ccc609'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:52.932407 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-134054531 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-134054531, it is now deprecated. Please use the default quota class for default quota. May 14 21:45:52.933205 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-134054531 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-134054531, it is now deprecated. Please use the default quota class for default quota. May 14 21:45:53.028877 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:53.105532 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:53.136404 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:45:53.143324 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] Deleted volume successfully. May 14 21:45:53.228780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Created reservations ['84c41a17-9fa5-4e58-a479-14c19295e743', '444fc674-8137-4dac-9fee-9bf7e7904dd9', '204106ec-6b90-44cf-bafe-f49109da8ab5', '9ac324ad-8785-48c6-8432-4e53bc349dad'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:53.238514 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-70c7783f-d006-4cbd-beec-58efa213a59f tempest-VolumesExtendTest-1486560883 None] Lock "b'cinder-22b0a87f-6086-493e-adf8-5e0b48222bf7-delete_volume'" released by "delete_volume" :: held 12.871s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:45:53.524661 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Created reservations ['a175593e-9401-4934-8ad7-7614b894d2a3', 'c83f8f88-67e4-42e4-8cfd-4fc57b4877dd', 'f3e1daa2-ad90-43e6-9778-d880964efe3e', '83e659ca-1459-4755-a8ae-d97fbd0859a7'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:45:53.557068 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Terminate volume connection completed successfully. May 14 21:45:53.574396 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-37b0f0d1-757c-43bd-b041-3b2be08496d9 tempest-VolumesBackupsTest-1778047115 None] Remove volume export completed successfully. May 14 21:45:53.951299 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:53.984140 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:54.026873 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:45:54.031901 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-32605e79-89a4-4acb-8df8-cd26f50cd3fd tempest-GroupsTest-1793587503 None] Delete group completed successfully. May 14 21:45:54.504660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-fe2b0773-b648-40ed-8148-08fa2937605a-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:45:54.591616 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:45:55.946742 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] Volume volume-fe2b0773-b648-40ed-8148-08fa2937605a is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:45:56.330465 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] deleting rbd volume volume-fe2b0773-b648-40ed-8148-08fa2937605a {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:46:00.717474 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-38af3133-f57b-4377-a439-3ff43b05bee2 tempest-GroupsV314Test-1330024577 None] Group tempest-GroupsV314Test-Group-1970096105: creating May 14 21:46:00.809054 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-38af3133-f57b-4377-a439-3ff43b05bee2 tempest-GroupsV314Test-1330024577 None] Group tempest-GroupsV314Test-Group-1970096105: created successfully May 14 21:46:00.819571 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-38af3133-f57b-4377-a439-3ff43b05bee2 tempest-GroupsV314Test-1330024577 None] Create group completed successfully. May 14 21:46:02.685461 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:46:02.705644 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Flow 'volume_create_manager' (ae7cff83-715c-49f9-9b48-ac6ef275dc7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:46:02.712895 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bee490b9-6f2d-4fc9-b43a-1d5f12b17f93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:02.728709 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bee490b9-6f2d-4fc9-b43a-1d5f12b17f93) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:46:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsV314Test-Volume-1134748616',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=2de05ee2-5234-4f7d-955b-638885de9658,host='ncdv-master-1@ceph#ceph',id=46f0706b-9886-41ca-82af-f91d75fda7b1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c2dae5c88af44b098436964aef153169',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:46:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:46:03Z,use_quota=True,user_id='a29e5f84202745cb8bd8f0fc78c5ced7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(fac6886e-190d-43d2-a77e-6f9131d0bc0a),volume_type_id=fac6886e-190d-43d2-a77e-6f9131d0bc0a)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:02.734253 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1f3fc461-c8ad-4499-800f-158b2e3dd6b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:02.738438 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1f3fc461-c8ad-4499-800f-158b2e3dd6b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:02.744193 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4694183a-84e4-433a-912c-482438d37e02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:02.748276 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4694183a-84e4-433a-912c-482438d37e02) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '46f0706b-9886-41ca-82af-f91d75fda7b1', 'volume_name': 'volume-46f0706b-9886-41ca-82af-f91d75fda7b1', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:02.753230 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d43a69b2-668e-4137-8ad3-f3de0109e6e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:02.757192 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d43a69b2-668e-4137-8ad3-f3de0109e6e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:02.761955 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (739f999f-27da-4ea7-b539-c23e160e306d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:02.763121 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Volume 46f0706b-9886-41ca-82af-f91d75fda7b1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-46f0706b-9886-41ca-82af-f91d75fda7b1', 'volume_size': 1} May 14 21:46:02.763537 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] creating volume 'volume-46f0706b-9886-41ca-82af-f91d75fda7b1' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:46:02.764331 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:03.880967 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:46:03.882253 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:46:03.886160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (739f999f-27da-4ea7-b539-c23e160e306d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-46f0706b-9886-41ca-82af-f91d75fda7b1', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:03.891170 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf8753bc-064b-4469-8f79-6013c6777558) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:03.946872 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Volume volume-46f0706b-9886-41ca-82af-f91d75fda7b1 (46f0706b-9886-41ca-82af-f91d75fda7b1): created successfully May 14 21:46:03.952452 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf8753bc-064b-4469-8f79-6013c6777558) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:03.958348 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Flow 'volume_create_manager' (ae7cff83-715c-49f9-9b48-ac6ef275dc7d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:46:04.021413 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-b34a9b9f-a9fc-41eb-94cc-8a95dab370ea tempest-GroupsV314Test-1330024577 None] Created volume successfully. May 14 21:46:04.838351 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] Created reservations ['e2892ed3-9c33-4d49-99ab-3f8ea320b28d', '4e2e84c6-70fb-45f8-a973-63799193b086', 'ab5deef7-b94a-4a81-a958-3d539497c4c6', '98a4335b-4927-4e96-88ed-3a629978ac28'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:04.917297 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:46:04.926487 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:04.997341 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:05.075421 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:46:05.222279 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:05.276753 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:05.332557 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:46:05.338053 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] Deleted volume successfully. May 14 21:46:05.425653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-16652eb0-fc95-4932-9d1d-9cd91f50dcf0 tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-fe2b0773-b648-40ed-8148-08fa2937605a-delete_volume'" released by "delete_volume" :: held 10.921s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:46:05.742240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:06.140600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] creating snapshot='volume-bd06f505-68d2-45e2-bb3a-f455fa4705e7.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:46:08.137263 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] cloning 'volume-46f0706b-9886-41ca-82af-f91d75fda7b1@volume-bd06f505-68d2-45e2-bb3a-f455fa4705e7.clone_snap' to 'volume-bd06f505-68d2-45e2-bb3a-f455fa4705e7' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:46:09.959168 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] Volume volume-46f0706b-9886-41ca-82af-f91d75fda7b1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:46:09.960891 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:46:09.961267 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:46:10.533420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:46:11.040713 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-73d0e3ca-22fb-410b-9408-e169fa3c24fb tempest-GroupsV314Test-1330024577 None] Create group from source-group-2de05ee2-5234-4f7d-955b-638885de9658 completed successfully. May 14 21:46:11.940411 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:12.212630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:46:12.418724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] deleting rbd volume volume-bd06f505-68d2-45e2-bb3a-f455fa4705e7 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:46:17.958843 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-af532b40-a505-4682-982e-e62144e2824c-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:46:18.039708 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:20.484552 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] deleting rbd volume volume-af532b40-a505-4682-982e-e62144e2824c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:46:20.489770 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:46:20.638021 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Volume volume-46f0706b-9886-41ca-82af-f91d75fda7b1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:46:20.638621 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] deleting parent snapshot volume-bd06f505-68d2-45e2-bb3a-f455fa4705e7.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:46:24.706300 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Created reservations ['0a255f5f-c6b1-4850-939e-0b9ac916f90a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:24.725545 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Deprecated: Default quota for resource: volumes_tempest-GroupsV314Test-volume-type-53005562 is set by the default quota flag: quota_volumes_tempest-GroupsV314Test-volume-type-53005562, it is now deprecated. Please use the default quota class for default quota. May 14 21:46:24.727058 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsV314Test-volume-type-53005562 is set by the default quota flag: quota_gigabytes_tempest-GroupsV314Test-volume-type-53005562, it is now deprecated. Please use the default quota class for default quota. May 14 21:46:24.810347 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Created reservations ['373fa7c7-6c3b-4a63-9db2-42dbbae7fef1', 'ed39e549-b09d-4392-9388-481f078f0614', 'd70bcef0-4027-4f9f-ad6b-c4370d3bb45d', '0a7c92d0-398d-44e2-8064-d27924136d0c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:25.207517 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:25.230396 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:25.253848 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:46:25.258591 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7e1f4869-e533-4f3f-97cb-5ccf103a7605 tempest-GroupsV314Test-1330024577 None] Delete group completed successfully. May 14 21:46:25.529557 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:46:25.650656 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] deleting parent snapshot volume-af532b40-a505-4682-982e-e62144e2824c.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:46:28.307797 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:28.454777 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:46:28.455283 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] Volume volume-46f0706b-9886-41ca-82af-f91d75fda7b1 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:46:28.540943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] deleting rbd volume volume-46f0706b-9886-41ca-82af-f91d75fda7b1 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:46:28.649363 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] Created reservations ['4888552b-6c6a-48bc-a238-9681c62ec9b4', '1ea2f585-6369-48bb-b863-904d71d2cfe9', 'a66777c7-1d36-4f14-b98f-23be3e1a39f7', '603b5f35-0b22-4c03-8f3d-45d9f6cfb63f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:28.803860 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:28.852320 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:28.875807 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:46:28.879746 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] Deleted volume successfully. May 14 21:46:28.938729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-94672387-c926-4cc6-b522-35be76bdd669 tempest-VolumesBackupsTest-1778047115 None] Lock "b'cinder-af532b40-a505-4682-982e-e62144e2824c-delete_volume'" released by "delete_volume" :: held 10.982s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:46:32.140925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] Created reservations ['94a031b5-2a4c-4326-9398-6883fe887eaa'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:32.231849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] Created reservations ['963246cd-1dc3-4bbe-bccc-ec752dee3593', 'dc36eb4b-c2a0-46b3-8efd-551bb7eb769b', '9519b5a5-5e04-4919-99c1-be455a14aa36', 'df596a50-4aa0-4c81-acd8-6e9e5c483727'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:32.533089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:32.572387 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:32.627315 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:46:32.631934 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2eee8361-4ee3-42aa-b533-e6bf5463c92c tempest-GroupsV314Test-1330024577 None] Delete group completed successfully. May 14 21:46:39.373347 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:46:39.400928 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Flow 'volume_create_manager' (a9855894-0d71-4c73-ba87-134d27e0bf1a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:46:39.411809 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (77c72b1a-12e1-4d3d-ab56-e9d88aa22d74) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:39.443346 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (77c72b1a-12e1-4d3d-ab56-e9d88aa22d74) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:46:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesAccessTest-Volume-1164358214',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=edfc18b3-dc60-4fdd-be99-b526ee927cee,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='82fb6149894440c1b5f3a8a8a2f5690f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:46:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:46:39Z,use_quota=True,user_id='46d03236b7c340f898f432bb5fb28f1b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(12b4e859-d857-4cf7-86ef-012063b58aae),volume_type_id=12b4e859-d857-4cf7-86ef-012063b58aae)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:39.449192 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c8dc3e2b-a21f-4bbf-95b0-10209c7beb47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:39.453500 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c8dc3e2b-a21f-4bbf-95b0-10209c7beb47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:39.458991 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1c10f28a-4a24-4924-97db-c5b933c4d1cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:39.463890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1c10f28a-4a24-4924-97db-c5b933c4d1cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'edfc18b3-dc60-4fdd-be99-b526ee927cee', 'volume_name': 'volume-edfc18b3-dc60-4fdd-be99-b526ee927cee', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:39.468972 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (88e154f3-d747-4cf0-b39e-2eefecb4bb88) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:39.473062 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (88e154f3-d747-4cf0-b39e-2eefecb4bb88) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:39.478145 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (22663f2e-b584-4afa-b3de-4561b1f472b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:39.481325 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Volume edfc18b3-dc60-4fdd-be99-b526ee927cee: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-edfc18b3-dc60-4fdd-be99-b526ee927cee', 'volume_size': 1} May 14 21:46:39.481325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] creating volume 'volume-edfc18b3-dc60-4fdd-be99-b526ee927cee' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:46:39.481325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:40.241403 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] extra_specs: {} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:46:40.242037 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] extra_specs: {} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:46:40.247617 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (22663f2e-b584-4afa-b3de-4561b1f472b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-edfc18b3-dc60-4fdd-be99-b526ee927cee', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:40.253019 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4903c4d0-82b4-4417-9f2d-0ed45e02d660) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:40.340163 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Volume volume-edfc18b3-dc60-4fdd-be99-b526ee927cee (edfc18b3-dc60-4fdd-be99-b526ee927cee): created successfully May 14 21:46:40.344205 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4903c4d0-82b4-4417-9f2d-0ed45e02d660) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:40.350233 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Flow 'volume_create_manager' (a9855894-0d71-4c73-ba87-134d27e0bf1a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:46:40.402688 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a2e6d268-2d30-42bb-be1b-65417a37d644 tempest-VolumeTypesAccessTest-832928980 None] Created volume successfully. May 14 21:46:42.176600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Lock "b'cinder-edfc18b3-dc60-4fdd-be99-b526ee927cee-delete_volume'" acquired by "delete_volume" :: waited 0.020s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:46:42.294801 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:42.465803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:46:42.466233 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Volume volume-edfc18b3-dc60-4fdd-be99-b526ee927cee is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:46:42.552516 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] deleting rbd volume volume-edfc18b3-dc60-4fdd-be99-b526ee927cee {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:46:45.838065 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-970377831 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-970377831, it is now deprecated. Please use the default quota class for default quota. May 14 21:46:45.838856 ncdv-master-1 cinder-volume[123870]: WARNING cinder.quota [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-970377831 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-970377831, it is now deprecated. Please use the default quota class for default quota. May 14 21:46:45.951754 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Created reservations ['3b4d1683-248e-4b10-967e-f28a05e0937f', 'f2fed09b-aab2-46e0-b30e-dbcc1871e1ee', 'e36de36e-d108-40fe-9e6c-a46829e79b82', '4ba094df-4a71-41b9-a5a3-c246c1d438e5'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:46:46.157892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:46.183042 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:46.209494 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:46:46.214353 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Deleted volume successfully. May 14 21:46:46.287565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-89548700-204b-4448-a81c-2014936034f7 tempest-VolumeTypesAccessTest-832928980 None] Lock "b'cinder-edfc18b3-dc60-4fdd-be99-b526ee927cee-delete_volume'" released by "delete_volume" :: held 4.111s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:46:57.095929 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:46:57.115756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Flow 'volume_create_manager' (32b93edd-f0c5-4a23-9f97-6f54d589ed23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:46:57.122646 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7ef663d3-b322-480b-ac32-67586c2da6f6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:57.139237 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7ef663d3-b322-480b-ac32-67586c2da6f6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:46:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-917585710',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ef327fb0-87a3-4d7a-8e78-931cec53ab4d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='70f3e8ff1b404e28aa96706c849a21e4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:46:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:46:57Z,use_quota=True,user_id='7ca5af3c599842d39df5edf9876c892a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:57.144571 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (663f8719-6766-4e3b-a832-4f487924b732) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:57.148927 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (663f8719-6766-4e3b-a832-4f487924b732) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:57.155080 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2aa015ff-cae0-49eb-a6bb-1948b4fbed61) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:57.159107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2aa015ff-cae0-49eb-a6bb-1948b4fbed61) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ef327fb0-87a3-4d7a-8e78-931cec53ab4d', 'volume_name': 'volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:57.164167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c76d3fdf-2c7c-480f-8586-93375a40e8ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:57.168576 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c76d3fdf-2c7c-480f-8586-93375a40e8ac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:57.173214 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (69be1f7d-1931-408e-8b8d-e3bbc17b524a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:57.174274 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Volume ef327fb0-87a3-4d7a-8e78-931cec53ab4d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d', 'volume_size': 1} May 14 21:46:57.174669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] creating volume 'volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:46:57.175436 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:46:57.800249 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:46:57.800635 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:46:57.804739 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (69be1f7d-1931-408e-8b8d-e3bbc17b524a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:57.809959 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b224d335-9472-47f6-94ca-4587bb355137) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:46:57.888654 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Volume volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d (ef327fb0-87a3-4d7a-8e78-931cec53ab4d): created successfully May 14 21:46:57.894541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b224d335-9472-47f6-94ca-4587bb355137) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:46:57.903089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Flow 'volume_create_manager' (32b93edd-f0c5-4a23-9f97-6f54d589ed23) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:46:57.968355 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-8d8ec9ce-65c5-4029-841b-1bf3610f0615 tempest-VolumesBackupsAdminTest-1826845199 None] Created volume successfully. May 14 21:46:59.576699 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:46:59Z,data_timestamp=2022-05-14T21:46:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-1246479446',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=80f8bfbe-cfe9-410c-a773-475f03a46ab4,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='70f3e8ff1b404e28aa96706c849a21e4',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='7ca5af3c599842d39df5edf9876c892a',volume_id=ef327fb0-87a3-4d7a-8e78-931cec53ab4d) May 14 21:46:59.577466 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] continue_backup in rpcapi backup_id 80f8bfbe-cfe9-410c-a773-475f03a46ab4 {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:46:59.684985 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:47:00.375131 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.690s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:47:00.376099 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'ef327fb0-87a3-4d7a-8e78-931cec53ab4d', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:47:00.391162 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] Initialize volume connection completed successfully. May 14 21:47:04.298187 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] Terminate volume connection completed successfully. May 14 21:47:04.314027 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2626f84c-e633-41ee-bd9b-0a3058dd281c tempest-VolumesBackupsAdminTest-1826845199 None] Remove volume export completed successfully. May 14 21:47:04.922699 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:47:04.924179 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:04.948174 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:04.972565 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:47:06.849743 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:47:06.871780 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Flow 'volume_create_manager' (36554f88-bf5d-41f3-bedd-c72a652afdff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:47:06.878762 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cf897778-479f-40f1-a05c-06282dfb8588) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:06.899022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cf897778-479f-40f1-a05c-06282dfb8588) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:47:06Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_80f8bfbe-cfe9-410c-a773-475f03a46ab4',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=71f49a8c-07af-4ed7-b50f-f0d553cd188e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='70f3e8ff1b404e28aa96706c849a21e4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:47:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:47:07Z,use_quota=True,user_id='7ca5af3c599842d39df5edf9876c892a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:06.904668 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f8b0c5f5-8aa4-4f70-bf59-942f3eb36676) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:06.908810 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f8b0c5f5-8aa4-4f70-bf59-942f3eb36676) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:06.913902 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5f924f5b-4553-4812-afc4-a21bf7501fe7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:06.919432 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5f924f5b-4553-4812-afc4-a21bf7501fe7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '71f49a8c-07af-4ed7-b50f-f0d553cd188e', 'volume_name': 'volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:06.923530 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (235a01a8-9780-4f35-8437-0d9584fc5fc9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:06.929762 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (235a01a8-9780-4f35-8437-0d9584fc5fc9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:06.935835 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (468aa463-a918-48da-a959-ef0b1681c939) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:06.936974 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Volume 71f49a8c-07af-4ed7-b50f-f0d553cd188e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e', 'volume_size': 1} May 14 21:47:06.937380 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] creating volume 'volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:47:06.938167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:07.434456 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:47:07.435066 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:47:07.438720 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (468aa463-a918-48da-a959-ef0b1681c939) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:07.443326 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8466e492-4eaf-45f7-aaa6-e5c2f5e6b1dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:07.539897 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Volume volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e (71f49a8c-07af-4ed7-b50f-f0d553cd188e): created successfully May 14 21:47:07.543858 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8466e492-4eaf-45f7-aaa6-e5c2f5e6b1dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:07.549554 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Flow 'volume_create_manager' (36554f88-bf5d-41f3-bedd-c72a652afdff) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:47:07.654481 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Created volume successfully. May 14 21:47:07.925375 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:47:08.549493 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.624s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:47:08.550315 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '71f49a8c-07af-4ed7-b50f-f0d553cd188e', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:47:08.566088 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Initialize volume connection completed successfully. May 14 21:47:11.040463 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Terminate volume connection completed successfully. May 14 21:47:11.056221 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-83ff4172-749b-4837-934e-6deed23cfb5c tempest-VolumesBackupsAdminTest-1826845199 None] Remove volume export completed successfully. May 14 21:47:12.209669 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] Lock "b'cinder-71f49a8c-07af-4ed7-b50f-f0d553cd188e-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:47:12.271884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:14.190651 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] Volume volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:47:14.660117 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] deleting rbd volume volume-71f49a8c-07af-4ed7-b50f-f0d553cd188e {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:47:19.420295 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] Created reservations ['6ae83d2c-762a-4c7d-919e-1acaf2905453', 'ba6b886a-7d87-4e78-8ba1-b515709ddbec', 'd984cb63-9134-4eb1-a5e2-569c436d8d1f', 'd470d420-28de-4c16-9592-66a482bf83e5'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:47:19.540061 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:19.562414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:19.585773 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:47:19.590111 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] Deleted volume successfully. May 14 21:47:19.652463 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-0496a2c7-124d-47b6-8e08-443207916ae0 tempest-VolumesBackupsAdminTest-1826845199 None] Lock "b'cinder-71f49a8c-07af-4ed7-b50f-f0d553cd188e-delete_volume'" released by "delete_volume" :: held 7.443s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:47:21.398386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:47:21.418571 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Flow 'volume_create_manager' (c094364e-4d1d-4520-8d85-1686b0d44d26) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:47:21.425204 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (36c54da9-9ba5-4298-bc89-aaa60eb09733) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:21.440626 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (36c54da9-9ba5-4298-bc89-aaa60eb09733) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:47:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1917907852',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fd86482c-c455-4c8d-84d0-f0da98fab425,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='70f3e8ff1b404e28aa96706c849a21e4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:47:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:47:21Z,use_quota=True,user_id='7ca5af3c599842d39df5edf9876c892a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:21.445979 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18867b8e-7b6d-4d2f-8c2b-e1a2970a3de1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:21.450081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18867b8e-7b6d-4d2f-8c2b-e1a2970a3de1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:21.455800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (65556579-2594-4c4c-8939-84186144154d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:21.459848 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (65556579-2594-4c4c-8939-84186144154d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fd86482c-c455-4c8d-84d0-f0da98fab425', 'volume_name': 'volume-fd86482c-c455-4c8d-84d0-f0da98fab425', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:21.464827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82efeefc-d06b-4069-b257-048f14c2b9ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:21.468705 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82efeefc-d06b-4069-b257-048f14c2b9ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:21.473386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aa916cbe-d0a0-494e-bae8-7a5499763f23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:21.474460 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Volume fd86482c-c455-4c8d-84d0-f0da98fab425: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fd86482c-c455-4c8d-84d0-f0da98fab425', 'volume_size': 1} May 14 21:47:21.474857 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] creating volume 'volume-fd86482c-c455-4c8d-84d0-f0da98fab425' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:47:21.475670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:21.925370 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:47:21.925686 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:47:21.929376 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aa916cbe-d0a0-494e-bae8-7a5499763f23) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fd86482c-c455-4c8d-84d0-f0da98fab425', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:21.933943 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a835b64-81a6-4822-9675-8d7f8b56b1d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:22.014115 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Volume volume-fd86482c-c455-4c8d-84d0-f0da98fab425 (fd86482c-c455-4c8d-84d0-f0da98fab425): created successfully May 14 21:47:22.018053 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3a835b64-81a6-4822-9675-8d7f8b56b1d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:22.023610 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Flow 'volume_create_manager' (c094364e-4d1d-4520-8d85-1686b0d44d26) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:47:22.082280 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-f4a65cd8-e05d-46f3-957d-6121ca1ff6aa tempest-VolumesBackupsAdminTest-1826845199 None] Created volume successfully. May 14 21:47:22.809025 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:47:22Z,data_timestamp=2022-05-14T21:47:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-1375944518',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=199436eb-6427-4066-a8d2-17a5f70f8025,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='70f3e8ff1b404e28aa96706c849a21e4',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='7ca5af3c599842d39df5edf9876c892a',volume_id=fd86482c-c455-4c8d-84d0-f0da98fab425) May 14 21:47:22.809849 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] continue_backup in rpcapi backup_id 199436eb-6427-4066-a8d2-17a5f70f8025 {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:47:22.915821 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:47:23.535734 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.619s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:47:23.536763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-fd86482c-c455-4c8d-84d0-f0da98fab425', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'fd86482c-c455-4c8d-84d0-f0da98fab425', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:47:23.552305 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] Initialize volume connection completed successfully. May 14 21:47:28.098678 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] Terminate volume connection completed successfully. May 14 21:47:28.115138 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ed8bdbf-e410-4aa6-abed-05911a707c22 tempest-VolumesBackupsAdminTest-1826845199 None] Remove volume export completed successfully. May 14 21:47:36.395912 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] Lock "b'cinder-fd86482c-c455-4c8d-84d0-f0da98fab425-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:47:36.477593 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:38.229662 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] Volume volume-fd86482c-c455-4c8d-84d0-f0da98fab425 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:47:38.494265 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] deleting rbd volume volume-fd86482c-c455-4c8d-84d0-f0da98fab425 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:47:41.522159 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] Created reservations ['f057d11a-50d6-46df-bf9d-ba848b9b6933', '836fec24-ab9d-4468-b2af-3fc772d6ae09', '00f238b8-aa38-4501-9d3f-14629fde6140', 'ef669ae1-4fae-4a29-9d81-e5b452cc8daa'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:47:41.673498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:41.696474 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:41.719747 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:47:41.724251 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] Deleted volume successfully. May 14 21:47:41.788741 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-96fe4162-c416-4e67-b89a-b89ad268714f tempest-VolumesBackupsAdminTest-1826845199 None] Lock "b'cinder-fd86482c-c455-4c8d-84d0-f0da98fab425-delete_volume'" released by "delete_volume" :: held 5.393s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:47:41.815439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] Lock "b'cinder-ef327fb0-87a3-4d7a-8e78-931cec53ab4d-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:47:41.881949 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:43.401968 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] Volume volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:47:43.699754 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] deleting rbd volume volume-ef327fb0-87a3-4d7a-8e78-931cec53ab4d {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:47:47.225488 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] Created reservations ['d9803822-552e-4833-bc22-41f98a7c21d8', '67b5a054-7e00-495e-bd36-01ca4958cc1d', 'eefe62e0-0ac4-4670-87cf-ca0921cc08a6', 'a6c32ee3-3362-4b29-9db2-2bd6c369607b'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:47:47.402738 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:47.427616 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:47.454153 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:47:47.462634 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] Deleted volume successfully. May 14 21:47:47.537140 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-05fa79bf-af50-4fbc-a78d-f206d87a68ef tempest-VolumesBackupsAdminTest-1826845199 None] Lock "b'cinder-ef327fb0-87a3-4d7a-8e78-931cec53ab4d-delete_volume'" released by "delete_volume" :: held 5.725s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:47:52.420655 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:47:52.440579 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Flow 'volume_create_manager' (49d072a5-fbc2-4c3e-a07f-37794fc17fe6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:47:52.447210 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (701e2b76-328b-41f5-9b51-580d4cac3277) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:52.463028 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (701e2b76-328b-41f5-9b51-580d4cac3277) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:47:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-1387749306',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=43252e67-043c-4010-8212-23c83dbb1cc9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a190ad103a11415991ae90d3aacbc330',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:47:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:47:52Z,use_quota=True,user_id='084b9518b50f44d99983915a32c3d9e6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:52.469584 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5c68c8f2-b58d-4999-873a-2d2a952433c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:52.473840 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5c68c8f2-b58d-4999-873a-2d2a952433c9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:52.479100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (54edb82b-b129-468f-aaaa-444fb86026da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:52.560685 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (54edb82b-b129-468f-aaaa-444fb86026da) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '43252e67-043c-4010-8212-23c83dbb1cc9', 'volume_name': 'volume-43252e67-043c-4010-8212-23c83dbb1cc9', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:52.565745 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8b1172c-2b00-4824-8e2b-06c27348a2ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:52.569995 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8b1172c-2b00-4824-8e2b-06c27348a2ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:52.574706 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e881f303-c7d1-41cb-a2b4-f90d2649288d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:52.576712 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Volume 43252e67-043c-4010-8212-23c83dbb1cc9: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-43252e67-043c-4010-8212-23c83dbb1cc9', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:47:52.577327 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Cloning 43252e67-043c-4010-8212-23c83dbb1cc9 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:47:52.585721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:47:52.664377 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 47, 52, 589692)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:47:52.664726 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:47:52.665168 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:47:52.669515 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.084s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:47:52.670438 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:47:52.676299 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 47, 52, 674038)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:47:52.676703 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:47:52.677089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:47:52.688508 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:52.829947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] creating snapshot='volume-43252e67-043c-4010-8212-23c83dbb1cc9.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:47:53.762234 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-43252e67-043c-4010-8212-23c83dbb1cc9.clone_snap' to 'volume-43252e67-043c-4010-8212-23c83dbb1cc9' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:47:54.564880 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:47:54.566163 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:47:54.566547 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:47:54.800743 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:47:54.802126 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Marking volume 43252e67-043c-4010-8212-23c83dbb1cc9 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:47:54.853276 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 43252e67-043c-4010-8212-23c83dbb1cc9. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:47:54.853763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Creating volume glance metadata for volume 43252e67-043c-4010-8212-23c83dbb1cc9 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:47:54.965505 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e881f303-c7d1-41cb-a2b4-f90d2649288d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-43252e67-043c-4010-8212-23c83dbb1cc9', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:54.970749 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bef57195-5528-4a40-8c35-8c4e208ee46d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:47:55.053147 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Volume volume-43252e67-043c-4010-8212-23c83dbb1cc9 (43252e67-043c-4010-8212-23c83dbb1cc9): created successfully May 14 21:47:55.056908 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bef57195-5528-4a40-8c35-8c4e208ee46d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:47:55.062539 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Flow 'volume_create_manager' (49d072a5-fbc2-4c3e-a07f-37794fc17fe6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:47:55.132987 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-a5fae629-ff5c-4a15-8dc7-7bd437d633d7 tempest-VolumesImageMetadata-501367317 None] Created volume successfully. May 14 21:47:55.822034 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] Lock "b'cinder-43252e67-043c-4010-8212-23c83dbb1cc9-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:47:55.916494 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:47:56.047660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:47:56.106005 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] deleting rbd volume volume-43252e67-043c-4010-8212-23c83dbb1cc9 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:47:58.745262 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:47:58.900061 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] deleting parent snapshot volume-43252e67-043c-4010-8212-23c83dbb1cc9.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:48:01.298619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] Created reservations ['e084ff72-b0b4-4acb-ad52-e9c1f44470ce', 'e27e9e7a-1f26-4234-be74-997259e61e13', 'e985fee9-6df5-4fc4-be58-464801f87a38', '159413b5-9d52-4414-a2c7-c835cefe3b61'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:48:01.447578 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:01.469212 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:01.492458 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:48:01.496811 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] Deleted volume successfully. May 14 21:48:01.559555 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-30495a08-f2df-4eb1-83e3-b322b66e06ad tempest-VolumesImageMetadata-501367317 None] Lock "b'cinder-43252e67-043c-4010-8212-23c83dbb1cc9-delete_volume'" released by "delete_volume" :: held 5.741s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:48:04.928877 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:48:04.930333 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:04.961904 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:04.987838 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:48:07.032029 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:48:07.051345 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Flow 'volume_create_manager' (c98ace48-4ea0-478b-8ccf-e188233924c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:48:07.057768 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9641462c-02b2-4d67-98ac-ae52ca0a8abf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:07.073612 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9641462c-02b2-4d67-98ac-ae52ca0a8abf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:48:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneNegativeTest-Volume-71367207',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d64d56db-31ce-4a5f-8430-eb41c00d7d1c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c9e643695d5540ce8147baa9a32dcd60',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:48:07Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:48:07Z,use_quota=True,user_id='54b3dc463aee4f19aac40890135a7c2c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:07.079925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3db0a5a9-bfc3-49b8-b212-f1ddf6c2958b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:07.083984 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3db0a5a9-bfc3-49b8-b212-f1ddf6c2958b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:07.089108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (064b6520-ec2a-42c1-b81f-45025a239a17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:07.093175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (064b6520-ec2a-42c1-b81f-45025a239a17) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd64d56db-31ce-4a5f-8430-eb41c00d7d1c', 'volume_name': 'volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c', 'volume_size': 2}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:07.098202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d7b3522c-af88-44af-a498-05c975b6ae0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:07.102069 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d7b3522c-af88-44af-a498-05c975b6ae0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:07.106764 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3a3bb8ca-f99a-4d05-a406-a7994e61da21) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:07.107879 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Volume d64d56db-31ce-4a5f-8430-eb41c00d7d1c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c', 'volume_size': 2} May 14 21:48:07.108269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] creating volume 'volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:48:07.109157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:07.496048 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:48:07.496430 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:48:07.500122 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3a3bb8ca-f99a-4d05-a406-a7994e61da21) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c', 'volume_size': 2}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:07.504705 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89cfd5e4-4f0b-4ba2-8fec-9a3f9bab1708) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:07.567021 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Volume volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c (d64d56db-31ce-4a5f-8430-eb41c00d7d1c): created successfully May 14 21:48:07.570577 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89cfd5e4-4f0b-4ba2-8fec-9a3f9bab1708) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:07.575958 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Flow 'volume_create_manager' (c98ace48-4ea0-478b-8ccf-e188233924c7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:48:07.630159 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-bed0a24d-292f-493b-9ade-271ff94b05d3 tempest-VolumesCloneNegativeTest-1442783057 None] Created volume successfully. May 14 21:48:08.046852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] Lock "b'cinder-d64d56db-31ce-4a5f-8430-eb41c00d7d1c-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:48:08.123729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:08.229778 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:48:08.230204 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] Volume volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:48:08.287765 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] deleting rbd volume volume-d64d56db-31ce-4a5f-8430-eb41c00d7d1c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:48:11.008108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] Created reservations ['a372a040-dbbe-4905-9861-322e60bdcb07', '0ab2dd88-e856-4414-b7ea-c4d9a6f5dd37', '82e26bb6-2e73-45da-a1b7-2e85cda5e940', '3119156a-fd61-4c32-81a2-b4b93134e4eb'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:48:11.150833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:11.174001 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:11.218454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:48:11.232358 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] Deleted volume successfully. May 14 21:48:11.342891 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-73c78635-a502-41cd-ac90-59477a7d3585 tempest-VolumesCloneNegativeTest-1442783057 None] Lock "b'cinder-d64d56db-31ce-4a5f-8430-eb41c00d7d1c-delete_volume'" released by "delete_volume" :: held 3.297s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:48:28.654243 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:48:28.679157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Flow 'volume_create_manager' (40ca966d-9944-49bd-a76f-81c41ab8f130) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:48:28.685850 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6bc192e2-56aa-436a-8611-b4dfc3ceb9af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:28.715522 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6bc192e2-56aa-436a-8611-b4dfc3ceb9af) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:48:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBackupRestore-volume-1327133213',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbd6e36a563a4245acae34e00437ca87',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:48:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:48:28Z,use_quota=True,user_id='1b51720c7970425abf4a26b2406893b7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:28.724815 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aedf3e34-fbfc-43c2-a4de-131935922514) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:28.733091 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aedf3e34-fbfc-43c2-a4de-131935922514) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:28.740193 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (652a495a-f1e5-420d-9e9c-0ea7840ace5f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:28.861171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (652a495a-f1e5-420d-9e9c-0ea7840ace5f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872', 'volume_name': 'volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:28.869766 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e84b7dfa-8e05-4ff2-91fa-bd47188aae8d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:28.870712 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e84b7dfa-8e05-4ff2-91fa-bd47188aae8d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:28.875466 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (03b3eb35-beb1-4ae4-a228-ae1b51c1d799) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:28.876699 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Volume f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:48:28.878172 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Cloning f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:48:28.894898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:48:28.974141 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 48, 28, 893791)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:48:28.974559 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:48:28.974964 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:48:28.981998 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.093s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:48:28.982879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:48:28.993319 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 48, 28, 987737)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:48:28.993982 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:48:28.994381 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:48:29.029756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:48:29.246777 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] creating snapshot='volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:48:31.161947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872.clone_snap' to 'volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:48:32.308266 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:48:32.309935 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:48:32.311348 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:48:32.595224 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:48:32.595870 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Marking volume f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:48:32.687353 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:48:32.687763 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Creating volume glance metadata for volume f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:48:32.833833 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (03b3eb35-beb1-4ae4-a228-ae1b51c1d799) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:32.838547 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (db63bbba-0708-4ba6-ba2d-048c440c7c8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:48:32.933760 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Volume volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872 (f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872): created successfully May 14 21:48:32.936991 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (db63bbba-0708-4ba6-ba2d-048c440c7c8e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:48:32.949107 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Flow 'volume_create_manager' (40ca966d-9944-49bd-a76f-81c41ab8f130) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:48:33.024956 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-2256cf73-ad1b-41a9-a15a-b8afb0c8a6e0 tempest-TestVolumeBackupRestore-1478043453 None] Created volume successfully. May 14 21:48:34.402509 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-14T21:48:33Z,data_timestamp=2022-05-14T21:48:33Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=0eb54d35-d84d-46e4-96c5-2f336ec4a342,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='fbd6e36a563a4245acae34e00437ca87',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='1b51720c7970425abf4a26b2406893b7',volume_id=f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872) May 14 21:48:34.403181 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.backup.rpcapi [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] continue_backup in rpcapi backup_id 0eb54d35-d84d-46e4-96c5-2f336ec4a342 {{(pid=123870) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 14 21:48:34.522092 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:48:35.204081 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.681s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:48:35.205089 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:48:35.226021 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] Initialize volume connection completed successfully. May 14 21:49:04.932963 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:49:04.934325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:49:04.958574 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:49:04.987118 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:49:17.119515 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] Terminate volume connection completed successfully. May 14 21:49:17.136062 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-18cf99ad-ac6f-40de-843e-b44a8e50d139 tempest-TestVolumeBackupRestore-1478043453 None] Remove volume export completed successfully. May 14 21:49:18.271004 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:49:18.291120 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Flow 'volume_create_manager' (a65ac792-20eb-430a-8709-1a586157bd43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:49:18.297724 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8674e9fd-8280-4784-a778-b980ea07d774) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:49:18.314647 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8674e9fd-8280-4784-a778-b980ea07d774) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:49:18Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_0eb54d35-d84d-46e4-96c5-2f336ec4a342',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e466666d-dad2-4f32-a5b6-875fbf06b6c9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fbd6e36a563a4245acae34e00437ca87',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:49:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:49:18Z,use_quota=True,user_id='1b51720c7970425abf4a26b2406893b7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:49:18.320125 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2f1c2119-b06a-4e39-a7a1-9c2c3a03ceaf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:49:18.324240 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2f1c2119-b06a-4e39-a7a1-9c2c3a03ceaf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:49:18.329325 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1fd05fc4-347d-44d6-b3c5-9af2b90ea620) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:49:18.334173 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1fd05fc4-347d-44d6-b3c5-9af2b90ea620) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'volume_name': 'volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:49:18.338890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1458af87-68b4-4e6e-95ef-4c24f8d6d336) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:49:18.342944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1458af87-68b4-4e6e-95ef-4c24f8d6d336) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:49:18.347853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c497bd4d-4409-44a0-b442-56323eff5918) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:49:18.348938 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Volume e466666d-dad2-4f32-a5b6-875fbf06b6c9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'volume_size': 1} May 14 21:49:18.349338 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] creating volume 'volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 21:49:18.350111 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:49:18.860553 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:49:18.862145 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:49:18.865547 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c497bd4d-4409-44a0-b442-56323eff5918) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:49:18.870060 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c89c89d4-7335-47ce-a3f2-ec2f6424a576) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:49:18.925481 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Volume volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9 (e466666d-dad2-4f32-a5b6-875fbf06b6c9): created successfully May 14 21:49:18.929130 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c89c89d4-7335-47ce-a3f2-ec2f6424a576) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:49:18.934805 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Flow 'volume_create_manager' (a65ac792-20eb-430a-8709-1a586157bd43) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:49:19.000476 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Created volume successfully. May 14 21:49:19.230123 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:49:19.863609 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.633s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:49:19.864434 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:49:19.880378 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Initialize volume connection completed successfully. May 14 21:50:03.107879 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Terminate volume connection completed successfully. May 14 21:50:03.123938 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-7bdc71ed-122f-4528-94fb-53081a2ad142 tempest-TestVolumeBackupRestore-1478043453 None] Remove volume export completed successfully. May 14 21:50:04.932834 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:50:04.936020 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:50:04.960032 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:50:04.985479 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:50:10.153886 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-99037a80-f1c1-45df-bcd4-77a9b62f9100 req-9dd6cde1-c6e3-404b-800d-9d19e89cdec6 tempest-TestVolumeBackupRestore-1478043453 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:50:10.851762 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-99037a80-f1c1-45df-bcd4-77a9b62f9100 req-9dd6cde1-c6e3-404b-800d-9d19e89cdec6 tempest-TestVolumeBackupRestore-1478043453 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.698s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:50:10.852852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-99037a80-f1c1-45df-bcd4-77a9b62f9100 req-9dd6cde1-c6e3-404b-800d-9d19e89cdec6 tempest-TestVolumeBackupRestore-1478043453 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:50:10.937583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-99037a80-f1c1-45df-bcd4-77a9b62f9100 req-9dd6cde1-c6e3-404b-800d-9d19e89cdec6 tempest-TestVolumeBackupRestore-1478043453 None] Connection info returned from driver {'name': 'volumes/volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e466666d-dad2-4f32-a5b6-875fbf06b6c9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '08b43cc6-ac3e-4fbb-9a27-cc49d72b117a'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:50:11.049487 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-99037a80-f1c1-45df-bcd4-77a9b62f9100 req-9dd6cde1-c6e3-404b-800d-9d19e89cdec6 tempest-TestVolumeBackupRestore-1478043453 None] attachment_update completed successfully. May 14 21:51:04.931297 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:51:04.932261 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:51:04.956891 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:51:04.980123 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:52:04.935922 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:52:04.937160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:52:04.979082 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:52:05.023740 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:53:04.938250 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:53:04.938925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:53:04.962516 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:53:04.985837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:54:04.941649 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:54:04.944410 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:54:04.976444 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:54:05.018653 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:55:04.939653 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:55:04.940889 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:55:04.964018 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:55:04.987425 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:55:44.377106 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-cf66932d-eecc-4ba3-8b50-1dc1351b6afc req-eedbc64d-f58a-4aac-9f20-c59465af1f0a tempest-TestVolumeBackupRestore-1478043453 None] Terminate volume connection completed successfully. May 14 21:55:44.377784 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-cf66932d-eecc-4ba3-8b50-1dc1351b6afc req-eedbc64d-f58a-4aac-9f20-c59465af1f0a tempest-TestVolumeBackupRestore-1478043453 None] Deleting attachment 08b43cc6-ac3e-4fbb-9a27-cc49d72b117a. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:55:46.688149 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] Lock "b'cinder-e466666d-dad2-4f32-a5b6-875fbf06b6c9-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:55:46.780101 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:55:49.555114 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] Volume volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:55:50.221753 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] deleting rbd volume volume-e466666d-dad2-4f32-a5b6-875fbf06b6c9 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:55:50.235369 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] Lock "b'cinder-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:55:50.342415 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:55:54.399617 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] deleting rbd volume volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:56:01.706867 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] Created reservations ['41ae93bb-45d8-417b-9374-462a30e1cc10', 'ad04e329-4a84-4a50-991b-f018bf3fae0b', '42ad9406-41b3-411b-94de-5ceacb40000b', 'f3d8165a-9b29-47a0-ac49-76f320b4dac4'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:56:01.843122 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:56:02.044795 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] deleting parent snapshot volume-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:56:03.995496 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:04.020721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:04.044498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:56:04.051878 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] Deleted volume successfully. May 14 21:56:04.135175 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] Created reservations ['f336cf86-e01b-4015-92c0-b69ff990370e', '60fbce52-aab9-4246-99c2-cf70c7d43885', '060eeebd-f04d-4076-bf16-8cc864a1d963', '013e986b-3e7d-45ba-8201-a039f08da310'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:56:04.157257 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5ec1fb94-8267-4409-b350-062bbc96bed6 tempest-TestVolumeBackupRestore-1478043453 None] Lock "b'cinder-e466666d-dad2-4f32-a5b6-875fbf06b6c9-delete_volume'" released by "delete_volume" :: held 17.469s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:56:04.395316 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:04.419216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:04.443491 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:56:04.449078 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] Deleted volume successfully. May 14 21:56:04.530121 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-94de5eda-416c-48be-9736-af97d8eb39d7 tempest-TestVolumeBackupRestore-1478043453 None] Lock "b'cinder-f3e8f8d6-a7b2-4a2b-9ca3-fda68b059872-delete_volume'" released by "delete_volume" :: held 14.295s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:56:04.943379 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:56:04.944117 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:04.965541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:04.988168 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:56:29.016370 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:56:29.037318 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (3bbaea5f-5de4-4ef2-8c0b-ac53cbe4cafc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:56:29.043925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b1bce295-6508-4698-91ee-fbb737842df6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:29.060275 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b1bce295-6508-4698-91ee-fbb737842df6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:56:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1164322580',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=12a2cdef-984d-4fa5-95b6-ad0989c6551b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9f801df18eb74b818c7ef4d2e10c576f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:56:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:56:29Z,use_quota=True,user_id='47332c347cf04b58b5857052c6468e6d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:29.065654 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2fe49ffd-177f-4a7b-b400-9ab31a29f4d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:29.069827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2fe49ffd-177f-4a7b-b400-9ab31a29f4d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:29.074869 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51a67961-b5b0-4461-9222-e19a21fb476a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:29.165697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51a67961-b5b0-4461-9222-e19a21fb476a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '12a2cdef-984d-4fa5-95b6-ad0989c6551b', 'volume_name': 'volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:29.170676 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0c7fa05d-f455-4fdc-9dc3-394155edaac4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:29.174872 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0c7fa05d-f455-4fdc-9dc3-394155edaac4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:29.180592 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f7417bc0-b186-4d85-8196-771c80abd2ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:29.181804 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Volume 12a2cdef-984d-4fa5-95b6-ad0989c6551b: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:56:29.182391 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Cloning 12a2cdef-984d-4fa5-95b6-ad0989c6551b from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:56:29.191650 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:56:29.257212 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 56, 29, 196660)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:56:29.257851 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:56:29.258333 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:56:29.263536 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.072s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:56:29.264478 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:56:29.271756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 56, 29, 269369)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:56:29.272246 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:56:29.272710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:56:29.285623 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:29.462897 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] creating snapshot='volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:56:30.783977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b.clone_snap' to 'volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:56:31.891237 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:56:31.893756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:56:31.893756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:56:32.188717 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:56:32.190059 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Marking volume 12a2cdef-984d-4fa5-95b6-ad0989c6551b as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:56:32.267428 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 12a2cdef-984d-4fa5-95b6-ad0989c6551b. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:56:32.269603 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Creating volume glance metadata for volume 12a2cdef-984d-4fa5-95b6-ad0989c6551b backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:56:32.425328 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f7417bc0-b186-4d85-8196-771c80abd2ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:32.430239 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8bbb9c63-928d-4280-9f72-689e1a4a0d1b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:32.494524 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Volume volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b (12a2cdef-984d-4fa5-95b6-ad0989c6551b): created successfully May 14 21:56:32.498167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8bbb9c63-928d-4280-9f72-689e1a4a0d1b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:32.504147 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (3bbaea5f-5de4-4ef2-8c0b-ac53cbe4cafc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:56:32.605987 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-d5f9201c-d474-4562-ae10-16f0b6561dfb tempest-TestVolumeBootPattern-1752998452 None] Created volume successfully. May 14 21:56:33.708925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-0ea14fed-8c65-4d01-883b-6451d628ba71 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:36.474462 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-0ea14fed-8c65-4d01-883b-6451d628ba71 tempest-TestVolumeBootPattern-1752998452 None] Create snapshot completed successfully May 14 21:56:44.230583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:56:44.257886 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (8c7dc544-af0c-4837-984c-72ddaabf5678) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:56:44.264336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b134c53b-5d78-4a13-9f70-7b7f3588144f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:44.280103 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b134c53b-5d78-4a13-9f70-7b7f3588144f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:56:44Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9f801df18eb74b818c7ef4d2e10c576f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:56:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=7cb873fb-cfc0-428f-9688-278942d461c2,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:56:44Z,use_quota=True,user_id='47332c347cf04b58b5857052c6468e6d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:44.286239 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9a5211ea-ac7a-4cd3-8e97-132b76466539) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:44.290617 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9a5211ea-ac7a-4cd3-8e97-132b76466539) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:44.295726 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74de4a23-b65b-42a9-8d49-f6b3e66a3193) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:44.299811 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74de4a23-b65b-42a9-8d49-f6b3e66a3193) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'volume_name': 'volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'volume_size': 1, 'snapshot_id': '7cb873fb-cfc0-428f-9688-278942d461c2'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:44.304700 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dfc5dc37-3251-4f7a-ae1b-e1dfa5522adb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:44.308593 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dfc5dc37-3251-4f7a-ae1b-e1dfa5522adb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:44.313853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9617d95d-d8bd-44a1-b798-d3cb0bcbb8cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:44.314430 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Volume b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'volume_size': 1, 'snapshot_id': '7cb873fb-cfc0-428f-9688-278942d461c2'} May 14 21:56:44.332979 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] cloning volumes/volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b@snapshot-7cb873fb-cfc0-428f-9688-278942d461c2 to volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c {{(pid=123870) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}} May 14 21:56:44.333670 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:44.394391 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:56:45.448261 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:56:45.448940 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:56:45.463747 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Marking volume b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:56:45.529101 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Copying metadata from snapshot 7cb873fb-cfc0-428f-9688-278942d461c2 to b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:427}} May 14 21:56:45.629329 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9617d95d-d8bd-44a1-b798-d3cb0bcbb8cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'volume_size': 1, 'snapshot_id': '7cb873fb-cfc0-428f-9688-278942d461c2'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:45.633934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d2188e54-da75-4d40-bde1-2b119225619e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:56:45.706025 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Volume volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c (b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c): created successfully May 14 21:56:45.709745 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d2188e54-da75-4d40-bde1-2b119225619e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:56:45.715246 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (8c7dc544-af0c-4837-984c-72ddaabf5678) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:56:45.814468 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-075d5049-fce6-42ff-a8d4-ac1f8cd7a8a8 tempest-TestVolumeBootPattern-1752998452 None] Created volume successfully. May 14 21:56:47.474741 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-f3d1c066-16b5-4ac4-9bc1-f1688f7a7c51 tempest-TestVolumeBootPattern-1752998452 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:56:48.160920 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-f3d1c066-16b5-4ac4-9bc1-f1688f7a7c51 tempest-TestVolumeBootPattern-1752998452 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.686s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:56:48.161757 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-f3d1c066-16b5-4ac4-9bc1-f1688f7a7c51 tempest-TestVolumeBootPattern-1752998452 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:56:48.251870 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-f3d1c066-16b5-4ac4-9bc1-f1688f7a7c51 tempest-TestVolumeBootPattern-1752998452 None] Connection info returned from driver {'name': 'volumes/volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '068f2b26-ea27-4bc9-bbf5-d43ccea6108e'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:56:48.358856 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-14b3fb10-545d-4836-a6bf-68a1245bf40e req-f3d1c066-16b5-4ac4-9bc1-f1688f7a7c51 tempest-TestVolumeBootPattern-1752998452 None] attachment_update completed successfully. May 14 21:57:04.942755 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:57:04.943808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:04.966762 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:04.989951 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:57:44.417320 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-cd7f8440-27cd-456a-aa13-056360215bbc req-8d0613e7-aafa-4a0c-801f-56106851bc28 tempest-TestVolumeBootPattern-1752998452 None] Terminate volume connection completed successfully. May 14 21:57:44.418031 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-cd7f8440-27cd-456a-aa13-056360215bbc req-8d0613e7-aafa-4a0c-801f-56106851bc28 tempest-TestVolumeBootPattern-1752998452 None] Deleting attachment 068f2b26-ea27-4bc9-bbf5-d43ccea6108e. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:57:44.752441 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:57:44.843586 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:45.075847 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:57:45.178003 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] deleting rbd volume volume-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:57:48.972927 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['3ff11b6f-697e-4217-981d-cfcc64caf737', 'fe7abc00-0f31-4a15-bf8b-6f934719d901', '6866ca24-1698-43d5-bd61-94d476259f6a', '19c7dd6f-ea3e-4c53-904a-cb9ddb56e3d5'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:57:49.136777 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:49.160086 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:49.182802 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:57:49.187647 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] Deleted volume successfully. May 14 21:57:49.259879 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-cd7f8440-27cd-456a-aa13-056360215bbc req-393d016e-f5ac-4004-9c69-944f7c7e888f tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b9b59cdf-23ee-4af9-ab1a-d5b9046c5b4c-delete_volume'" released by "delete_volume" :: held 4.507s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:57:49.934968 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9c44d77b-8a4d-40fc-b452-9ae3dbd4c9fb tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-7cb873fb-cfc0-428f-9688-278942d461c2-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:57:49.934968 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-9c44d77b-8a4d-40fc-b452-9ae3dbd4c9fb tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:51.950552 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-9c44d77b-8a4d-40fc-b452-9ae3dbd4c9fb tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['eb0fde95-da34-4f13-b7a2-75b143ca3225', '5cb8aa83-197c-4d94-b571-329b773ff908', '0b4af60f-f0e2-40f1-ad79-a28f6c82fb4f', '632e8e70-7e4c-4725-8d79-5cc0f723a2bc'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:57:52.227993 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-9c44d77b-8a4d-40fc-b452-9ae3dbd4c9fb tempest-TestVolumeBootPattern-1752998452 None] Delete snapshot completed successfully. May 14 21:57:52.232823 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-9c44d77b-8a4d-40fc-b452-9ae3dbd4c9fb tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-7cb873fb-cfc0-428f-9688-278942d461c2-delete_snapshot'" released by "delete_snapshot" :: held 2.301s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:57:53.262862 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-12a2cdef-984d-4fa5-95b6-ad0989c6551b-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:57:53.367828 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:53.561462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:57:53.711761 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] deleting rbd volume volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:57:56.776844 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:57:56.914282 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] deleting parent snapshot volume-12a2cdef-984d-4fa5-95b6-ad0989c6551b.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:57:59.573058 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['37ab6e36-a24c-4734-8826-24ac45c9851a', '1ba909a6-c763-4e58-acc3-5a5dad1b6d6b', '0f02c216-c2a4-403d-92ff-a13e2eac2b49', 'd3a8a11c-411a-4e2f-9115-08e028366e0f'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:57:59.754125 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:59.780662 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:57:59.947932 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:57:59.952718 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] Deleted volume successfully. May 14 21:58:00.040710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-3c65b1a1-85e9-4d81-8ba3-950bb4460022 tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-12a2cdef-984d-4fa5-95b6-ad0989c6551b-delete_volume'" released by "delete_volume" :: held 6.778s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:58:01.535350 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:58:01.559254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (fdaf4ed4-3c27-45df-8258-5136d62b58bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:58:01.567027 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f407104a-5fa8-4d94-9839-dc40a8edef60) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:01.585944 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f407104a-5fa8-4d94-9839-dc40a8edef60) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:58:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1991410706',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8eb810f7-e5ef-40f5-9a22-95d2637dd9d7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9f801df18eb74b818c7ef4d2e10c576f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:58:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:58:01Z,use_quota=True,user_id='47332c347cf04b58b5857052c6468e6d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:01.591732 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (946bfae6-8e19-4200-92a4-bdf0b6d43e81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:01.600671 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (946bfae6-8e19-4200-92a4-bdf0b6d43e81) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:01.602934 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (99345f82-cb2c-4206-a4da-90564c8a830b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:01.695322 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (99345f82-cb2c-4206-a4da-90564c8a830b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'volume_name': 'volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:01.700535 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac6dd963-a526-46b0-9751-33951483c45d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:01.704561 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac6dd963-a526-46b0-9751-33951483c45d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:01.709529 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (63f96671-5f2d-4f01-9022-93b28786bc12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:01.710543 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Volume 8eb810f7-e5ef-40f5-9a22-95d2637dd9d7: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:58:01.711121 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Cloning 8eb810f7-e5ef-40f5-9a22-95d2637dd9d7 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:58:01.720277 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.008s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:58:01.796340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 58, 1, 724489)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:58:01.796896 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:58:01.798495 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:58:01.805164 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.085s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:58:01.806269 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:58:01.813775 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 58, 1, 809897)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:58:01.814213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:58:01.814881 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:58:01.826746 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:02.018536 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] creating snapshot='volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:58:03.336648 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7.clone_snap' to 'volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:58:04.393767 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:58:04.394886 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:58:04.395345 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:58:04.648310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:58:04.648852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Marking volume 8eb810f7-e5ef-40f5-9a22-95d2637dd9d7 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:58:04.738602 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to 8eb810f7-e5ef-40f5-9a22-95d2637dd9d7. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:58:04.739037 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Creating volume glance metadata for volume 8eb810f7-e5ef-40f5-9a22-95d2637dd9d7 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:58:04.839411 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (63f96671-5f2d-4f01-9022-93b28786bc12) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:04.844956 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d9d6d9b3-aa62-4a98-b73b-4bd1ed04c8d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:04.934735 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Volume volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7 (8eb810f7-e5ef-40f5-9a22-95d2637dd9d7): created successfully May 14 21:58:04.938598 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d9d6d9b3-aa62-4a98-b73b-4bd1ed04c8d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:04.944256 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (fdaf4ed4-3c27-45df-8258-5136d62b58bd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:58:04.948056 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:58:04.948056 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:05.056905 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:05.085109 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:58:05.178772 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-c1eb9524-4b51-4e25-9294-a5aecb6afe2e tempest-TestVolumeBootPattern-1752998452 None] Created volume successfully. May 14 21:58:08.521252 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-5db897ea-25ce-4271-8baf-d0f2a8dad28f req-a170d715-baa4-48f2-a5d3-0d31708436f9 tempest-TestVolumeBootPattern-1752998452 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:58:09.176901 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-5db897ea-25ce-4271-8baf-d0f2a8dad28f req-a170d715-baa4-48f2-a5d3-0d31708436f9 tempest-TestVolumeBootPattern-1752998452 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.655s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:58:09.177765 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-5db897ea-25ce-4271-8baf-d0f2a8dad28f req-a170d715-baa4-48f2-a5d3-0d31708436f9 tempest-TestVolumeBootPattern-1752998452 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:58:09.247405 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-5db897ea-25ce-4271-8baf-d0f2a8dad28f req-a170d715-baa4-48f2-a5d3-0d31708436f9 tempest-TestVolumeBootPattern-1752998452 None] Connection info returned from driver {'name': 'volumes/volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8eb810f7-e5ef-40f5-9a22-95d2637dd9d7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'bc313625-b370-4390-8963-c90c0a6aadc3'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:58:09.350164 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-5db897ea-25ce-4271-8baf-d0f2a8dad28f req-a170d715-baa4-48f2-a5d3-0d31708436f9 tempest-TestVolumeBootPattern-1752998452 None] attachment_update completed successfully. May 14 21:58:19.142979 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-26ab8d53-9766-453c-9939-48fcf8662c09 req-0ac49ef4-0836-46d4-bb6e-cd8792d24611 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:20.697918 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-26ab8d53-9766-453c-9939-48fcf8662c09 req-0ac49ef4-0836-46d4-bb6e-cd8792d24611 tempest-TestVolumeBootPattern-1752998452 None] Create snapshot completed successfully May 14 21:58:24.951114 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:58:24.979622 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (c9e94e9b-f879-4373-b404-eab026c4c9ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:58:24.991179 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cd988726-8fa1-41a8-8c49-0a18cf61bfca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:25.002979 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cd988726-8fa1-41a8-8c49-0a18cf61bfca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:58:24Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8375266c-5b8c-4af0-8d95-1178d5ebf302,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9f801df18eb74b818c7ef4d2e10c576f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:58:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=b9d7842a-919c-4b43-b206-f44b760eb6d7,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:58:25Z,use_quota=True,user_id='47332c347cf04b58b5857052c6468e6d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:25.008619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51a21eed-52cc-4004-affa-ed3bf6d85dcc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:25.012784 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51a21eed-52cc-4004-affa-ed3bf6d85dcc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:25.018481 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (de161450-8b0f-45a1-9871-7079c7bf9666) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:25.022634 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (de161450-8b0f-45a1-9871-7079c7bf9666) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '8375266c-5b8c-4af0-8d95-1178d5ebf302', 'volume_name': 'volume-8375266c-5b8c-4af0-8d95-1178d5ebf302', 'volume_size': 1, 'snapshot_id': 'b9d7842a-919c-4b43-b206-f44b760eb6d7'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:25.027564 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b701b356-03f8-4b62-9b51-0fdc89e39232) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:25.032336 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b701b356-03f8-4b62-9b51-0fdc89e39232) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:25.037071 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (25678acb-3dcc-43e4-9a9d-1d2f0b4e9650) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:25.038162 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Volume 8375266c-5b8c-4af0-8d95-1178d5ebf302: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-8375266c-5b8c-4af0-8d95-1178d5ebf302', 'volume_size': 1, 'snapshot_id': 'b9d7842a-919c-4b43-b206-f44b760eb6d7'} May 14 21:58:25.059076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] cloning volumes/volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7@snapshot-b9d7842a-919c-4b43-b206-f44b760eb6d7 to volume-8375266c-5b8c-4af0-8d95-1178d5ebf302 {{(pid=123870) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}} May 14 21:58:25.059782 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:25.123246 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:26.099013 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:58:26.102868 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:58:26.122819 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Marking volume 8375266c-5b8c-4af0-8d95-1178d5ebf302 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:58:26.221232 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Copying metadata from snapshot b9d7842a-919c-4b43-b206-f44b760eb6d7 to 8375266c-5b8c-4af0-8d95-1178d5ebf302. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:427}} May 14 21:58:26.318750 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (25678acb-3dcc-43e4-9a9d-1d2f0b4e9650) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8375266c-5b8c-4af0-8d95-1178d5ebf302', 'volume_size': 1, 'snapshot_id': 'b9d7842a-919c-4b43-b206-f44b760eb6d7'}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:26.323414 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a080c418-0b1b-402a-8f2b-7f37060a61b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:58:26.400827 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Volume volume-8375266c-5b8c-4af0-8d95-1178d5ebf302 (8375266c-5b8c-4af0-8d95-1178d5ebf302): created successfully May 14 21:58:26.404813 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a080c418-0b1b-402a-8f2b-7f37060a61b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:58:26.411248 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (c9e94e9b-f879-4373-b404-eab026c4c9ae) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:58:26.470107 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-88896067-44a4-4882-8021-e9ce5345aa06 tempest-TestVolumeBootPattern-1752998452 None] Created volume successfully. May 14 21:58:28.178445 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-dd586535-d201-4a68-8221-5fa3b5b06d94 tempest-TestVolumeBootPattern-1752998452 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:58:28.940661 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-dd586535-d201-4a68-8221-5fa3b5b06d94 tempest-TestVolumeBootPattern-1752998452 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.762s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:58:28.941518 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-dd586535-d201-4a68-8221-5fa3b5b06d94 tempest-TestVolumeBootPattern-1752998452 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8375266c-5b8c-4af0-8d95-1178d5ebf302', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '8375266c-5b8c-4af0-8d95-1178d5ebf302', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:58:29.046991 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-dd586535-d201-4a68-8221-5fa3b5b06d94 tempest-TestVolumeBootPattern-1752998452 None] Connection info returned from driver {'name': 'volumes/volume-8375266c-5b8c-4af0-8d95-1178d5ebf302', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8375266c-5b8c-4af0-8d95-1178d5ebf302', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'dcb52288-3264-4dee-ba49-949a8b7317a9'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:58:29.168335 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-16a2739d-6b74-4ca2-be0c-b318eb0bb6ea req-dd586535-d201-4a68-8221-5fa3b5b06d94 tempest-TestVolumeBootPattern-1752998452 None] attachment_update completed successfully. May 14 21:58:41.796381 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-74ce7c51-dd2c-4291-ba18-1c1d4e90b6ef tempest-TestVolumeBootPattern-1752998452 None] Terminate volume connection completed successfully. May 14 21:58:41.797097 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-74ce7c51-dd2c-4291-ba18-1c1d4e90b6ef tempest-TestVolumeBootPattern-1752998452 None] Deleting attachment dcb52288-3264-4dee-ba49-949a8b7317a9. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:58:42.206756 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-8375266c-5b8c-4af0-8d95-1178d5ebf302-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:58:42.296576 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:42.835076 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:58:43.085931 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] deleting rbd volume volume-8375266c-5b8c-4af0-8d95-1178d5ebf302 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:58:49.749102 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['69d2d9d3-c576-4284-ac73-e03980a2eafc', 'd345ca85-88bd-4bd6-952e-d578f3f68653', '68eb0673-1909-4433-bcc1-f88bae547c9b', 'cc70d88a-a01d-484a-9aa7-65647ab4631a'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:58:49.925539 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:49.947959 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:58:49.970189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:58:49.975356 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] Deleted volume successfully. May 14 21:58:50.082686 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-24bed171-e1e1-4c80-b883-df0f022f0a1d req-f0815e25-0df9-42d8-86b9-cd4469fd4d9c tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-8375266c-5b8c-4af0-8d95-1178d5ebf302-delete_volume'" released by "delete_volume" :: held 7.876s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:58:50.533883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-19f55f20-469e-4831-b0e4-98aa51c1ab4d tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b9d7842a-919c-4b43-b206-f44b760eb6d7-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:58:50.536463 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-19f55f20-469e-4831-b0e4-98aa51c1ab4d tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:04.951101 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 21:59:04.951884 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:04.974399 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:04.996419 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:59:21.912707 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-19f55f20-469e-4831-b0e4-98aa51c1ab4d tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['841b04a4-fc4c-4e97-8000-08edd6d48b17', '10be2a0c-5b84-4b45-b802-6da172e75052', 'a178f91a-ade3-43e9-8d11-038df5d65c86', '53134c3c-080b-4954-a608-51b5dd000c01'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:59:22.141024 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-19f55f20-469e-4831-b0e4-98aa51c1ab4d tempest-TestVolumeBootPattern-1752998452 None] Delete snapshot completed successfully. May 14 21:59:22.145408 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-19f55f20-469e-4831-b0e4-98aa51c1ab4d tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b9d7842a-919c-4b43-b206-f44b760eb6d7-delete_snapshot'" released by "delete_snapshot" :: held 31.612s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:59:25.272445 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-cdf5426e-c311-4780-80f8-1df305665da1 req-33670c76-c9ea-4e2d-ba46-2a9707015edf tempest-TestVolumeBootPattern-1752998452 None] Terminate volume connection completed successfully. May 14 21:59:25.273618 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-cdf5426e-c311-4780-80f8-1df305665da1 req-33670c76-c9ea-4e2d-ba46-2a9707015edf tempest-TestVolumeBootPattern-1752998452 None] Deleting attachment bc313625-b370-4390-8963-c90c0a6aadc3. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 21:59:25.563074 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:59:25.667680 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:25.957384 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 21:59:26.142187 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] deleting rbd volume volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 21:59:29.777925 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 21:59:29.895992 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] deleting parent snapshot volume-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 21:59:31.666287 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['5e255a58-be34-45ee-a58e-52ff39bf15d4', 'f68a671f-5c68-4bd4-b613-f159c958b2dc', '7a48253c-f790-460f-8ace-e6028fd47763', 'acadda38-c6a4-4563-a07a-8d74104ee83c'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 21:59:31.804236 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:31.825366 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:31.850299 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 21:59:31.855618 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] Deleted volume successfully. May 14 21:59:31.930278 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-cdf5426e-c311-4780-80f8-1df305665da1 req-93f27648-9c95-402d-a800-f9ac42776ad6 tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-8eb810f7-e5ef-40f5-9a22-95d2637dd9d7-delete_volume'" released by "delete_volume" :: held 6.367s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:59:36.822698 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 21:59:36.843297 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (426f528f-d319-4205-ad97-b616dfb9d30f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:59:36.849874 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a118d5a5-f685-4eba-a79c-75de85268ecc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:59:36.864837 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a118d5a5-f685-4eba-a79c-75de85268ecc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T21:59:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-11398316',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b001bcfd-7605-45a8-bc29-58aed9008929,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9f801df18eb74b818c7ef4d2e10c576f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T21:59:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T21:59:37Z,use_quota=True,user_id='47332c347cf04b58b5857052c6468e6d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:59:36.870290 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (250031d2-f364-4c0f-b0c9-d7c41e248213) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:59:36.874542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (250031d2-f364-4c0f-b0c9-d7c41e248213) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:59:36.879759 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5347e893-2376-45d9-b8ec-243842905b12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:59:36.960976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5347e893-2376-45d9-b8ec-243842905b12) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'b001bcfd-7605-45a8-bc29-58aed9008929', 'volume_name': 'volume-b001bcfd-7605-45a8-bc29-58aed9008929', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:59:36.966012 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f68d3126-3484-4f55-b281-4c5609490904) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:59:36.969949 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f68d3126-3484-4f55-b281-4c5609490904) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:59:36.974781 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4c723a4e-078d-48ef-a488-a17ce3c10473) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:59:36.976011 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Volume b001bcfd-7605-45a8-bc29-58aed9008929: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-b001bcfd-7605-45a8-bc29-58aed9008929', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 21:59:36.976583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Cloning b001bcfd-7605-45a8-bc29-58aed9008929 from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 21:59:36.984786 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.007s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 21:59:37.037561 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 59, 36, 989373)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:59:37.038022 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:59:37.038426 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 21:59:37.042761 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.058s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 21:59:37.043698 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 21:59:37.049420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 21, 59, 37, 46955)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 21:59:37.051171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 21:59:37.051171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 21:59:37.061808 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 21:59:37.191739 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] creating snapshot='volume-b001bcfd-7605-45a8-bc29-58aed9008929.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 21:59:38.137165 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-b001bcfd-7605-45a8-bc29-58aed9008929.clone_snap' to 'volume-b001bcfd-7605-45a8-bc29-58aed9008929' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 21:59:39.152029 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 21:59:39.153109 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 21:59:39.153480 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 21:59:39.435660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 21:59:39.436318 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Marking volume b001bcfd-7605-45a8-bc29-58aed9008929 as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 21:59:39.519159 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to b001bcfd-7605-45a8-bc29-58aed9008929. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 21:59:39.519600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Creating volume glance metadata for volume b001bcfd-7605-45a8-bc29-58aed9008929 backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 21:59:39.632227 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4c723a4e-078d-48ef-a488-a17ce3c10473) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b001bcfd-7605-45a8-bc29-58aed9008929', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:59:39.636802 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2d46e317-0606-4ebb-ab16-9c3584ee8958) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 21:59:39.723229 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Volume volume-b001bcfd-7605-45a8-bc29-58aed9008929 (b001bcfd-7605-45a8-bc29-58aed9008929): created successfully May 14 21:59:39.727765 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2d46e317-0606-4ebb-ab16-9c3584ee8958) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 21:59:39.733164 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Flow 'volume_create_manager' (426f528f-d319-4205-ad97-b616dfb9d30f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 21:59:39.808974 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-696c2827-1de0-435e-be8d-89dc5eab28a6 tempest-TestVolumeBootPattern-1752998452 None] Created volume successfully. May 14 21:59:44.172814 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-6d5a57ed-8095-45d7-bf0f-298284a55edf req-edbbcc89-f6ea-490c-8071-ebdf1cabbcca tempest-TestVolumeBootPattern-1752998452 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 21:59:44.838552 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-6d5a57ed-8095-45d7-bf0f-298284a55edf req-edbbcc89-f6ea-490c-8071-ebdf1cabbcca tempest-TestVolumeBootPattern-1752998452 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.666s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 21:59:44.839310 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-6d5a57ed-8095-45d7-bf0f-298284a55edf req-edbbcc89-f6ea-490c-8071-ebdf1cabbcca tempest-TestVolumeBootPattern-1752998452 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b001bcfd-7605-45a8-bc29-58aed9008929', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'b001bcfd-7605-45a8-bc29-58aed9008929', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 21:59:44.942898 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-6d5a57ed-8095-45d7-bf0f-298284a55edf req-edbbcc89-f6ea-490c-8071-ebdf1cabbcca tempest-TestVolumeBootPattern-1752998452 None] Connection info returned from driver {'name': 'volumes/volume-b001bcfd-7605-45a8-bc29-58aed9008929', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b001bcfd-7605-45a8-bc29-58aed9008929', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5e0ada62-0cb6-4883-b1f7-e277e2547ed3'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 21:59:45.062843 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-6d5a57ed-8095-45d7-bf0f-298284a55edf req-edbbcc89-f6ea-490c-8071-ebdf1cabbcca tempest-TestVolumeBootPattern-1752998452 None] attachment_update completed successfully. May 14 22:00:04.951106 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:00:04.952176 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:00:04.986140 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:00:05.019434 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:01:04.955881 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:01:04.956664 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:01:04.983264 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:01:05.015540 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:02:04.960468 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:02:04.961213 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:02:04.984664 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:02:05.013446 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:03:04.959029 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:03:04.959769 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:03:04.981529 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:03:05.005177 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:03:19.813589 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-029e2608-ccee-49ef-a60b-835d6a70e54e req-9da83df6-3d01-42a0-b33f-7facf99f48d0 tempest-TestVolumeBootPattern-1752998452 None] Terminate volume connection completed successfully. May 14 22:03:19.814273 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-029e2608-ccee-49ef-a60b-835d6a70e54e req-9da83df6-3d01-42a0-b33f-7facf99f48d0 tempest-TestVolumeBootPattern-1752998452 None] Deleting attachment 5e0ada62-0cb6-4883-b1f7-e277e2547ed3. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 22:03:21.433252 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b001bcfd-7605-45a8-bc29-58aed9008929-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 22:03:21.521660 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:03:21.698042 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 22:03:21.814012 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] deleting rbd volume volume-b001bcfd-7605-45a8-bc29-58aed9008929 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 22:03:25.264865 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 22:03:25.373198 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] deleting parent snapshot volume-b001bcfd-7605-45a8-bc29-58aed9008929.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 22:03:27.967193 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] Created reservations ['5b7c1a00-9c39-433c-82c1-52f34edcbcf6', '4968b42b-44c2-4f7c-98c7-7f9b4e9e9e2d', '3a96f61e-4b22-48bc-8d9b-073b8107dd12', 'e792ad0d-8b64-4b30-905e-3cd1ab291fdc'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 22:03:28.144789 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:03:28.210710 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:03:28.233899 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:03:28.238108 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] Deleted volume successfully. May 14 22:03:28.325221 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cfdc7d47-74c8-4733-acf6-305a3abce37c tempest-TestVolumeBootPattern-1752998452 None] Lock "b'cinder-b001bcfd-7605-45a8-bc29-58aed9008929-delete_volume'" released by "delete_volume" :: held 6.892s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 22:04:04.959962 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:04:04.960625 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:04:04.983093 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:04:05.006137 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:05:04.962623 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:05:04.963331 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:05:04.991054 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:05:05.021657 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:06:04.967305 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:06:04.968020 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:06:04.989484 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:06:05.012188 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:07:04.968160 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:07:04.968843 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:07:04.992224 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:07:05.015274 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:08:04.971682 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:08:04.973141 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:08:04.996729 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:08:05.021189 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:09:04.976469 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:09:04.977148 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:09:04.998561 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:09:05.021520 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:09:50.196454 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 22:09:50.216341 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Flow 'volume_create_manager' (2b409f8a-3966-416b-b488-b13c449b2448) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 22:09:50.225942 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fc17ba46-5ae8-4d1e-8ea6-d952728948cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:09:50.240369 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fc17ba46-5ae8-4d1e-8ea6-d952728948cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T22:09:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerActionsTestJSON-volume-1076980203',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6fe194cc-792c-4fc5-aef4-bfc0c2f55c17,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5b7eff5698214fe982348f81c276dc6a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T22:09:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T22:09:50Z,use_quota=True,user_id='58d7a44fb4b64d5d98bdda07c1638f20',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:09:50.245731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (50787418-8638-41a0-a0c3-22941a785629) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:09:50.249859 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (50787418-8638-41a0-a0c3-22941a785629) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:09:50.255581 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5abef95e-5250-4723-91a9-4700c145df29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:09:50.259615 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5abef95e-5250-4723-91a9-4700c145df29) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'volume_name': 'volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:09:50.264533 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (93e242cd-32a3-4bfa-8729-6af8a5bfbbd1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:09:50.268966 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (93e242cd-32a3-4bfa-8729-6af8a5bfbbd1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:09:50.273439 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (63cc23a7-37d0-4851-b7e8-b0208deea308) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:09:50.275097 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Volume 6fe194cc-792c-4fc5-aef4-bfc0c2f55c17: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'volume_size': 1} May 14 22:09:50.275519 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] creating volume 'volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 22:09:50.276326 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:09:50.728158 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 22:09:50.728529 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 22:09:50.732201 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (63cc23a7-37d0-4851-b7e8-b0208deea308) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:09:50.737343 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (766f0d0b-dc76-4895-ab84-65abb02839eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:09:50.806196 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Volume volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17 (6fe194cc-792c-4fc5-aef4-bfc0c2f55c17): created successfully May 14 22:09:50.809782 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (766f0d0b-dc76-4895-ab84-65abb02839eb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:09:50.816093 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Flow 'volume_create_manager' (2b409f8a-3966-416b-b488-b13c449b2448) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 22:09:50.885945 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5dafb148-dcb1-428c-a95f-8905b38a3af9 tempest-ServerActionsTestJSON-1153994297 None] Created volume successfully. May 14 22:09:51.984168 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-620972ba-bc94-4442-b82e-99a8266b3c4b req-a67e35b5-3112-4b7b-9647-02f3d8a54be9 tempest-ServerActionsTestJSON-1153994297 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 22:09:52.680053 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-620972ba-bc94-4442-b82e-99a8266b3c4b req-a67e35b5-3112-4b7b-9647-02f3d8a54be9 tempest-ServerActionsTestJSON-1153994297 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.695s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 22:09:52.680878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-620972ba-bc94-4442-b82e-99a8266b3c4b req-a67e35b5-3112-4b7b-9647-02f3d8a54be9 tempest-ServerActionsTestJSON-1153994297 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 22:09:52.766606 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-620972ba-bc94-4442-b82e-99a8266b3c4b req-a67e35b5-3112-4b7b-9647-02f3d8a54be9 tempest-ServerActionsTestJSON-1153994297 None] Connection info returned from driver {'name': 'volumes/volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e3e6b05c-ce80-4429-be6a-224098ffe669'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 22:09:52.881445 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-620972ba-bc94-4442-b82e-99a8266b3c4b req-a67e35b5-3112-4b7b-9647-02f3d8a54be9 tempest-ServerActionsTestJSON-1153994297 None] attachment_update completed successfully. May 14 22:10:04.976741 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:10:04.977672 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:10:05.001420 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:10:05.024208 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:10:15.735025 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-d23961bc-b755-43be-aa6e-bc5a9b5e1c22 tempest-ServerActionsTestJSON-1153994297 None] Terminate volume connection completed successfully. May 14 22:10:15.735697 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-d23961bc-b755-43be-aa6e-bc5a9b5e1c22 tempest-ServerActionsTestJSON-1153994297 None] Deleting attachment e3e6b05c-ce80-4429-be6a-224098ffe669. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 22:10:18.637850 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-79311670-5f8c-4a7e-8219-7f15adcf912e tempest-ServerActionsTestJSON-1153994297 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 22:10:19.261464 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-79311670-5f8c-4a7e-8219-7f15adcf912e tempest-ServerActionsTestJSON-1153994297 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.623s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 22:10:19.262194 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-79311670-5f8c-4a7e-8219-7f15adcf912e tempest-ServerActionsTestJSON-1153994297 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 22:10:19.360541 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-79311670-5f8c-4a7e-8219-7f15adcf912e tempest-ServerActionsTestJSON-1153994297 None] Connection info returned from driver {'name': 'volumes/volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '2f2a23f8-cb57-43eb-b275-7397c391fc4e'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 22:10:19.481684 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-2acd74ea-4b7f-481b-80eb-67ce603397ac req-79311670-5f8c-4a7e-8219-7f15adcf912e tempest-ServerActionsTestJSON-1153994297 None] attachment_update completed successfully. May 14 22:10:27.255422 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-eacf2271-a220-4051-969e-620e49bb8d8c tempest-ServerActionsTestJSON-1153994297 None] Terminate volume connection completed successfully. May 14 22:10:27.256155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-eacf2271-a220-4051-969e-620e49bb8d8c tempest-ServerActionsTestJSON-1153994297 None] Deleting attachment 2f2a23f8-cb57-43eb-b275-7397c391fc4e. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 22:10:30.183572 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-1ade09f1-56b3-45bd-ab49-94e2e24d47fa tempest-ServerActionsTestJSON-1153994297 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 22:10:30.822581 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-1ade09f1-56b3-45bd-ab49-94e2e24d47fa tempest-ServerActionsTestJSON-1153994297 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.639s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 22:10:30.823404 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-1ade09f1-56b3-45bd-ab49-94e2e24d47fa tempest-ServerActionsTestJSON-1153994297 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 22:10:30.915976 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-1ade09f1-56b3-45bd-ab49-94e2e24d47fa tempest-ServerActionsTestJSON-1153994297 None] Connection info returned from driver {'name': 'volumes/volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6fe194cc-792c-4fc5-aef4-bfc0c2f55c17', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0b9309f6-5f96-4c7a-85ed-09cf4ed9db81'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 22:10:31.039439 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-bc714dce-7c09-47b9-bdae-12eaa3ba1b60 req-1ade09f1-56b3-45bd-ab49-94e2e24d47fa tempest-ServerActionsTestJSON-1153994297 None] attachment_update completed successfully. May 14 22:10:38.021462 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-d11ac74d-f575-44b7-989a-b67dbece18ad req-3dd5fab1-c946-4b4b-b76c-e7929cb9cbd1 tempest-ServerActionsTestJSON-1153994297 None] Terminate volume connection completed successfully. May 14 22:10:38.022157 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-d11ac74d-f575-44b7-989a-b67dbece18ad req-3dd5fab1-c946-4b4b-b76c-e7929cb9cbd1 tempest-ServerActionsTestJSON-1153994297 None] Deleting attachment 0b9309f6-5f96-4c7a-85ed-09cf4ed9db81. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 22:10:39.203630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] Lock "b'cinder-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 22:10:39.298745 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:10:39.491117 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 22:10:39.491645 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] Volume volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 22:10:39.657492 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] deleting rbd volume volume-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 22:10:42.707695 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] Created reservations ['db5a0b65-8641-47eb-902b-e3626f7e48cb', '4f21fbe0-c078-48d9-ab6d-3a7223856020', 'efa32944-b2c6-42eb-bafc-f8cf552c4e88', '6c2c79c6-6689-4ca2-bc4b-a06c36239bc3'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 22:10:42.880176 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:10:42.907878 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:10:42.934981 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:10:42.939269 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] Deleted volume successfully. May 14 22:10:43.025584 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-5b29099a-0f73-4958-ba33-a8ffd0044de2 tempest-ServerActionsTestJSON-1153994297 None] Lock "b'cinder-6fe194cc-792c-4fc5-aef4-bfc0c2f55c17-delete_volume'" released by "delete_volume" :: held 3.822s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 22:11:04.979123 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:11:04.979966 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:11:05.010630 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:11:05.039052 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:12:35.703160 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 22:12:35.722975 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Flow 'volume_create_manager' (1cb28681-1c59-4e0f-892f-e57972dfe731) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 22:12:35.729677 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (271bac48-646a-48ab-876f-75b82674d286) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:12:35.745293 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (271bac48-646a-48ab-876f-75b82674d286) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T22:12:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-257963928',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d9ae9734-da55-464a-9282-3d7c60ec4f22,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='86b19a25559c4ce7a8a6ee7c86fcab56',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T22:12:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T22:12:36Z,use_quota=True,user_id='db572732af5342faacfc59e0841f68bf',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:12:35.750689 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aed58fb1-5cda-44d4-91bc-c4eaa5083709) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:12:35.754832 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aed58fb1-5cda-44d4-91bc-c4eaa5083709) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:12:35.761778 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6fe8e6d6-35a7-41a4-a67a-2635510eb5c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:12:35.767970 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6fe8e6d6-35a7-41a4-a67a-2635510eb5c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd9ae9734-da55-464a-9282-3d7c60ec4f22', 'volume_name': 'volume-d9ae9734-da55-464a-9282-3d7c60ec4f22', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:12:35.773344 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8bb4b471-ccff-4c8e-b7f5-15260bcc3b59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:12:35.777267 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8bb4b471-ccff-4c8e-b7f5-15260bcc3b59) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:12:35.781977 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (267c0809-f514-40f3-94d6-57fd13135ef2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:12:35.783075 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Volume d9ae9734-da55-464a-9282-3d7c60ec4f22: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d9ae9734-da55-464a-9282-3d7c60ec4f22', 'volume_size': 1} May 14 22:12:35.783497 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] creating volume 'volume-d9ae9734-da55-464a-9282-3d7c60ec4f22' {{(pid=123870) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}} May 14 22:12:35.784288 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:12:36.337242 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 22:12:36.337539 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 22:12:36.341523 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (267c0809-f514-40f3-94d6-57fd13135ef2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d9ae9734-da55-464a-9282-3d7c60ec4f22', 'volume_size': 1}' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:12:36.346202 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b5bd7a5a-d264-4eec-8769-bf82cf7f15ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:12:36.398298 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Volume volume-d9ae9734-da55-464a-9282-3d7c60ec4f22 (d9ae9734-da55-464a-9282-3d7c60ec4f22): created successfully May 14 22:12:36.401903 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b5bd7a5a-d264-4eec-8769-bf82cf7f15ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:12:36.407377 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Flow 'volume_create_manager' (1cb28681-1c59-4e0f-892f-e57972dfe731) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 22:12:36.455609 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-646f2d30-74c8-47a6-b8a7-ad3b96faadd1 tempest-TestMinimumBasicScenario-598422200 None] Created volume successfully. May 14 22:12:37.521064 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-1015e7f0-b8e6-434c-bdea-76055539cd18 req-99f4db04-5382-4f87-89b4-a1f1c64def9f tempest-TestMinimumBasicScenario-598422200 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 22:12:38.335301 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-1015e7f0-b8e6-434c-bdea-76055539cd18 req-99f4db04-5382-4f87-89b4-a1f1c64def9f tempest-TestMinimumBasicScenario-598422200 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.814s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 22:12:38.336842 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-1015e7f0-b8e6-434c-bdea-76055539cd18 req-99f4db04-5382-4f87-89b4-a1f1c64def9f tempest-TestMinimumBasicScenario-598422200 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d9ae9734-da55-464a-9282-3d7c60ec4f22', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'd9ae9734-da55-464a-9282-3d7c60ec4f22', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 22:12:38.415800 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-1015e7f0-b8e6-434c-bdea-76055539cd18 req-99f4db04-5382-4f87-89b4-a1f1c64def9f tempest-TestMinimumBasicScenario-598422200 None] Connection info returned from driver {'name': 'volumes/volume-d9ae9734-da55-464a-9282-3d7c60ec4f22', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd9ae9734-da55-464a-9282-3d7c60ec4f22', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '9aead3a0-3e13-42a5-b0ba-27997cd513ff'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 22:12:38.542488 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-1015e7f0-b8e6-434c-bdea-76055539cd18 req-99f4db04-5382-4f87-89b4-a1f1c64def9f tempest-TestMinimumBasicScenario-598422200 None] attachment_update completed successfully. May 14 22:13:04.985329 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:13:04.986259 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:13:05.009853 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:13:05.031642 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:14:01.717777 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-a42e65d5-9f4b-4633-97df-29474575a2e2 req-2d9438a2-8121-4e14-af38-83e82e8d85b5 tempest-TestMinimumBasicScenario-598422200 None] Terminate volume connection completed successfully. May 14 22:14:01.718672 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-a42e65d5-9f4b-4633-97df-29474575a2e2 req-2d9438a2-8121-4e14-af38-83e82e8d85b5 tempest-TestMinimumBasicScenario-598422200 None] Deleting attachment 9aead3a0-3e13-42a5-b0ba-27997cd513ff. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 22:14:02.510100 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] Lock "b'cinder-d9ae9734-da55-464a-9282-3d7c60ec4f22-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 22:14:02.602827 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:14:02.785553 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 22:14:02.786183 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] Volume volume-d9ae9734-da55-464a-9282-3d7c60ec4f22 is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 22:14:02.906238 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] deleting rbd volume volume-d9ae9734-da55-464a-9282-3d7c60ec4f22 {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 22:14:04.986684 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:14:04.987368 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:14:05.008758 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:14:05.032018 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:14:05.995194 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] Created reservations ['08519247-8dd6-4159-9658-4de890fac511', '4751fa78-9a2e-483b-9497-82102626b67b', '245a6565-e625-41f6-83d8-a8536c6a5a8c', '18747952-4696-4bbd-a461-07cd9add01eb'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 22:14:06.202301 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:14:06.224014 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:14:06.246595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:14:06.251350 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] Deleted volume successfully. May 14 22:14:06.320947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-cb5459ea-7814-424a-80cb-943f6442c9e4 tempest-TestMinimumBasicScenario-598422200 None] Lock "b'cinder-d9ae9734-da55-464a-9282-3d7c60ec4f22-delete_volume'" released by "delete_volume" :: held 3.811s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 22:15:04.987010 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:15:04.987747 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:15:05.009474 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:15:05.031598 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:16:04.987315 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:16:04.988014 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:16:05.010001 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:16:05.032978 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:17:04.987498 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:17:04.988171 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:17:05.014514 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:17:05.037250 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:18:04.982882 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:18:04.983590 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:18:05.006504 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:18:05.028595 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:19:04.983410 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:19:04.984167 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:19:05.014862 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:19:05.062081 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:20:04.987567 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:20:04.988256 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:20:05.010906 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:20:05.033675 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:21:04.988105 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:21:04.991728 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:21:05.035904 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:21:05.071364 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:22:04.990908 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:22:04.991655 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:22:05.018470 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:22:05.041684 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:23:04.987117 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:23:04.987823 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:23:05.010428 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:23:05.034248 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:24:04.989401 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:24:04.990115 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:24:05.011779 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:24:05.034721 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:25:04.989712 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:25:04.991509 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:25:05.028855 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:25:05.062627 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:26:04.986193 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:26:04.986869 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:26:05.010232 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:26:05.031883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:26:47.113910 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=123870) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1348}} May 14 22:26:47.133498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Flow 'volume_create_manager' (412116ce-304c-46e5-a4b5-4f80b4aa2a26) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 22:26:47.139973 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (723505a6-f546-4ac4-aad0-8d2f2996ff4b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:26:47.154987 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (723505a6-f546-4ac4-aad0-8d2f2996ff4b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-14T22:26:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1843428003',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e821e22e-91d9-4342-8936-37eac17a4aee,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a339e577d4bb4a4d8bc280b4edab9ef8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-14T22:26:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-14T22:26:47Z,use_quota=True,user_id='b3ec4b155d344282909459de93091eee',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0a4caf03-4613-4ca9-9e3b-1297a183a0fc),volume_type_id=0a4caf03-4613-4ca9-9e3b-1297a183a0fc)' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:26:47.160844 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e1386e07-4b58-458f-9e59-e79eaa9c3bcd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:26:47.164969 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e1386e07-4b58-458f-9e59-e79eaa9c3bcd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:26:47.170216 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bdf9d4dc-d7c4-44c0-9ad5-64b01fb681a0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:26:47.256561 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bdf9d4dc-d7c4-44c0-9ad5-64b01fb681a0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'e821e22e-91d9-4342-8936-37eac17a4aee', 'volume_name': 'volume-e821e22e-91d9-4342-8936-37eac17a4aee', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:26:47.261598 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (806902f4-179a-4af7-94a4-7d5004137661) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:26:47.265616 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (806902f4-179a-4af7-94a4-7d5004137661) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:26:47.270552 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (631af388-15f3-40e0-a4fe-faee1cae9346) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:26:47.272012 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Volume e821e22e-91d9-4342-8936-37eac17a4aee: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-e821e22e-91d9-4342-8936-37eac17a4aee', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 14 22:26:47.272619 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Cloning e821e22e-91d9-4342-8936-37eac17a4aee from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e at location (None, None). {{(pid=123870) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1051}} May 14 22:26:47.280727 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" acquired by "_prepare_image_cache_entry" :: waited 0.007s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 22:26:47.322883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 22, 26, 47, 285530)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 22:26:47.323309 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 22:26:47.323740 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Found cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:874}} May 14 22:26:47.328072 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Lock "b'cinder-b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e'" released by "_prepare_image_cache_entry" :: held 0.047s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 22:26:47.330839 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Attempting to retrieve cache entry for image = b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e on host ncdv-master-1@ceph#ceph. {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:815}} May 14 22:26:47.334986 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Found image-volume cache entry: {'id': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'volume_id': '59ff51c3-dc23-4af9-af80-0fb5c11ea498', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30), 'last_used': datetime.datetime(2022, 5, 14, 22, 26, 47, 332761)}. {{(pid=123870) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 14 22:26:47.335427 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.image.cache [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Image-volume cache entry image_update_at = 2022-05-14 21:02:30+00:00, requested image updated_at = 2022-05-14 21:02:30+00:00. {{(pid=123870) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 14 22:26:47.335892 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Creating from source image-volume 59ff51c3-dc23-4af9-af80-0fb5c11ea498 {{(pid=123870) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:829}} May 14 22:26:47.347890 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:26:47.482636 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] creating snapshot='volume-e821e22e-91d9-4342-8936-37eac17a4aee.clone_snap' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}} May 14 22:26:48.449115 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] cloning 'volume-59ff51c3-dc23-4af9-af80-0fb5c11ea498@volume-e821e22e-91d9-4342-8936-37eac17a4aee.clone_snap' to 'volume-e821e22e-91d9-4342-8936-37eac17a4aee' {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}} May 14 22:26:49.229803 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Volume volume-a73964bc-4e69-4eef-b351-fcd6ec89ade1.deleted is not a clone. {{(pid=123870) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}} May 14 22:26:49.230804 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}} May 14 22:26:49.231183 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=123870) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}} May 14 22:26:49.486897 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] clone created successfully {{(pid=123870) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}} May 14 22:26:49.488155 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.volume_utils [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Marking volume e821e22e-91d9-4342-8936-37eac17a4aee as bootable. {{(pid=123870) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1129}} May 14 22:26:49.545548 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Copying metadata from image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e to e821e22e-91d9-4342-8936-37eac17a4aee. {{(pid=123870) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:448}} May 14 22:26:49.545971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Creating volume glance metadata for volume e821e22e-91d9-4342-8936-37eac17a4aee backed by image b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=123870) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:671}} May 14 22:26:49.651924 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (631af388-15f3-40e0-a4fe-faee1cae9346) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e821e22e-91d9-4342-8936-37eac17a4aee', 'volume_size': 1, 'image_id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '96a027574eeb4536a94751bde54cf3b1', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e', 'created_at': datetime.datetime(2022, 5, 14, 21, 2, 13, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 14, 21, 2, 30, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/b1b30a17-4fb4-4d05-9f8c-cb82afc1e28e/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:26:49.656600 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac0eb278-0a6b-46c7-a70d-fdf145ac1d0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 14 22:26:49.734067 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.flows.manager.create_volume [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Volume volume-e821e22e-91d9-4342-8936-37eac17a4aee (e821e22e-91d9-4342-8936-37eac17a4aee): created successfully May 14 22:26:49.737731 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac0eb278-0a6b-46c7-a70d-fdf145ac1d0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=123870) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 14 22:26:49.743230 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Flow 'volume_create_manager' (412116ce-304c-46e5-a4b5-4f80b4aa2a26) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=123870) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 14 22:26:49.819870 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [None req-10a3a739-9049-4a95-a313-22e309d90311 tempest-TestShelveInstance-207090090 None] Created volume successfully. May 14 22:26:53.038257 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-0531aa92-b2e1-4b5b-b8fb-82f264923e1a req-84d17e71-042f-4987-af3a-b6304673ac0d tempest-TestShelveInstance-207090090 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 14 22:26:53.739777 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_concurrency.processutils [req-0531aa92-b2e1-4b5b-b8fb-82f264923e1a req-84d17e71-042f-4987-af3a-b6304673ac0d tempest-TestShelveInstance-207090090 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.701s {{(pid=123870) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 14 22:26:53.740914 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-0531aa92-b2e1-4b5b-b8fb-82f264923e1a req-84d17e71-042f-4987-af3a-b6304673ac0d tempest-TestShelveInstance-207090090 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e821e22e-91d9-4342-8936-37eac17a4aee', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b3399bd5-b147-45ea-b0ac-77d4339d7906', 'volume_id': 'e821e22e-91d9-4342-8936-37eac17a4aee', 'discard': True}} {{(pid=123870) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1652}} May 14 22:26:53.820772 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-0531aa92-b2e1-4b5b-b8fb-82f264923e1a req-84d17e71-042f-4987-af3a-b6304673ac0d tempest-TestShelveInstance-207090090 None] Connection info returned from driver {'name': 'volumes/volume-e821e22e-91d9-4342-8936-37eac17a4aee', 'hosts': ['10.222.0.52'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e821e22e-91d9-4342-8936-37eac17a4aee', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ed18c940-918f-4c93-96fa-6f6f883ad459'} {{(pid=123870) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4824}} May 14 22:26:53.948975 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-0531aa92-b2e1-4b5b-b8fb-82f264923e1a req-84d17e71-042f-4987-af3a-b6304673ac0d tempest-TestShelveInstance-207090090 None] attachment_update completed successfully. May 14 22:27:04.987336 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:27:04.988304 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:27:05.012542 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:27:05.035254 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:28:04.984187 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:28:04.984883 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:28:05.006866 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:28:05.029717 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:29:04.985338 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:29:04.986025 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:29:05.008229 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:29:05.031047 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:30:04.985815 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:30:04.986499 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:30:05.008129 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:30:05.030387 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:30:26.853559 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-387656e8-0128-495a-a689-7b9e96122350 tempest-TestShelveInstance-207090090 None] Terminate volume connection completed successfully. May 14 22:30:26.854186 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.manager [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-387656e8-0128-495a-a689-7b9e96122350 tempest-TestShelveInstance-207090090 None] Deleting attachment ed18c940-918f-4c93-96fa-6f6f883ad459. {{(pid=123870) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4955}} May 14 22:30:27.180583 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] Lock "b'cinder-e821e22e-91d9-4342-8936-37eac17a4aee-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 14 22:30:27.303715 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:30:27.552855 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] volume has no backup snaps {{(pid=123870) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}} May 14 22:30:27.647324 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] deleting rbd volume volume-e821e22e-91d9-4342-8936-37eac17a4aee {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1327}} May 14 22:30:32.058852 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] volume is a clone so cleaning references {{(pid=123870) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1348}} May 14 22:30:32.281132 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] deleting parent snapshot volume-e821e22e-91d9-4342-8936-37eac17a4aee.clone_snap {{(pid=123870) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}} May 14 22:30:34.177779 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.quota [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] Created reservations ['881669eb-980a-4404-a8fb-a10bab0fe7c2', '9c08d01a-e888-4dcf-8e4a-c925c91dbd75', '851f2616-2f53-4d23-b187-a905f9abbbaa', '9c5c2948-e815-4beb-a17a-b27bc31bb081'] {{(pid=123870) reserve /opt/stack/cinder/cinder/quota.py:755}} May 14 22:30:34.348046 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:30:34.402894 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:30:34.453359 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:30:34.459052 ncdv-master-1 cinder-volume[123870]: INFO cinder.volume.manager [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] Deleted volume successfully. May 14 22:30:34.519628 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.coordination [req-ffc65520-af2a-41c3-8c45-1a93f9f31033 req-a46bc3dd-38dc-4402-b460-a33459995e98 tempest-TestShelveInstance-207090090 None] Lock "b'cinder-e821e22e-91d9-4342-8936-37eac17a4aee-delete_volume'" released by "delete_volume" :: held 7.339s {{(pid=123870) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 14 22:31:04.986189 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:31:04.986861 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:31:05.008532 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:31:05.031512 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:32:04.986260 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:32:04.986929 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:32:05.008560 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:32:05.031077 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:33:04.985798 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:33:04.986480 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:33:05.035687 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:33:05.060551 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:34:04.986210 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:34:04.986867 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:34:05.015332 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:34:05.036537 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:35:04.991752 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:35:04.992462 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:35:05.014666 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:35:05.039023 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:36:04.994363 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:36:04.995082 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:36:05.026971 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:36:05.049340 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:37:04.994602 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:37:04.995285 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:37:05.021476 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:37:05.050306 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:38:04.995809 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:38:04.996498 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:38:05.043108 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:38:05.064765 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:39:04.999723 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:39:05.000386 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:39:05.022026 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:39:05.044540 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:40:04.999381 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:40:05.000075 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:40:05.022607 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:40:05.045871 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:41:05.000870 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:41:05.001562 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:41:05.023257 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:41:05.045804 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:42:05.001134 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:42:05.001947 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:42:05.030517 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:42:05.058650 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:43:05.004391 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:43:05.005032 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:43:05.026578 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:43:05.048086 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 14 22:44:05.003599 ncdv-master-1 cinder-volume[123870]: DEBUG oslo_service.periodic_task [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=123870) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 14 22:44:05.004270 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:44:05.027143 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.volume.drivers.rbd [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=123870) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:524}} May 14 22:44:05.050395 ncdv-master-1 cinder-volume[123870]: DEBUG cinder.manager [None req-5717dcf3-6733-4fad-9af9-5280cbc16806 None None] Notifying Schedulers of capabilities ... {{(pid=123870) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}